-- Logs begin at Fri 2022-08-05 14:36:10 UTC, end at Fri 2022-08-05 15:49:31 UTC. -- Aug 05 15:11:20.178875 dv-cinder-849890-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 05 15:11:20.191592 dv-cinder-849890-4-iscsi devstack@keystone.service[80534]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 05 15:11:20.198247 dv-cinder-849890-4-iscsi devstack@keystone.service[80534]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Aug 5 15:11:20 2022] *** Aug 05 15:11:20.198247 dv-cinder-849890-4-iscsi devstack@keystone.service[80534]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 05 15:11:20.198247 dv-cinder-849890-4-iscsi devstack@keystone.service[80534]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 05 15:11:20.198247 dv-cinder-849890-4-iscsi devstack@keystone.service[80534]: nodename: dv-cinder-849890-4-iscsi Aug 05 15:11:20.198247 dv-cinder-849890-4-iscsi devstack@keystone.service[80534]: machine: x86_64 Aug 05 15:11:20.198247 dv-cinder-849890-4-iscsi devstack@keystone.service[80534]: clock source: unix Aug 05 15:11:20.198247 dv-cinder-849890-4-iscsi devstack@keystone.service[80534]: pcre jit disabled Aug 05 15:11:20.198247 dv-cinder-849890-4-iscsi devstack@keystone.service[80534]: detected number of CPU cores: 4 Aug 05 15:11:20.198247 dv-cinder-849890-4-iscsi devstack@keystone.service[80534]: current working directory: / Aug 05 15:11:20.198247 dv-cinder-849890-4-iscsi devstack@keystone.service[80534]: detected binary path: /usr/bin/uwsgi-core Aug 05 15:11:20.198247 dv-cinder-849890-4-iscsi devstack@keystone.service[80534]: your processes number limit is 47680 Aug 05 15:11:20.198247 dv-cinder-849890-4-iscsi devstack@keystone.service[80534]: your memory page size is 4096 bytes Aug 05 15:11:20.198247 dv-cinder-849890-4-iscsi devstack@keystone.service[80534]: detected max file descriptor number: 2048 Aug 05 15:11:20.198247 dv-cinder-849890-4-iscsi devstack@keystone.service[80534]: lock engine: pthread robust mutexes Aug 05 15:11:20.198247 dv-cinder-849890-4-iscsi devstack@keystone.service[80534]: thunder lock: enabled Aug 05 15:11:20.198247 dv-cinder-849890-4-iscsi devstack@keystone.service[80534]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 05 15:11:20.198247 dv-cinder-849890-4-iscsi devstack@keystone.service[80534]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 05 15:11:20.239832 dv-cinder-849890-4-iscsi devstack@keystone.service[80534]: Python main interpreter initialized at 0x55d197d249c0 Aug 05 15:11:20.239832 dv-cinder-849890-4-iscsi devstack@keystone.service[80534]: python threads support enabled Aug 05 15:11:20.239832 dv-cinder-849890-4-iscsi devstack@keystone.service[80534]: your server socket listen backlog is limited to 100 connections Aug 05 15:11:20.239832 dv-cinder-849890-4-iscsi devstack@keystone.service[80534]: your mercy for graceful operations on workers is 90 seconds Aug 05 15:11:20.240272 dv-cinder-849890-4-iscsi devstack@keystone.service[80534]: mapped 403077 bytes (393 KB) for 2 cores Aug 05 15:11:20.240272 dv-cinder-849890-4-iscsi devstack@keystone.service[80534]: *** Operational MODE: preforking *** Aug 05 15:11:20.240572 dv-cinder-849890-4-iscsi devstack@keystone.service[80534]: *** uWSGI is running in multiple interpreter mode *** Aug 05 15:11:20.240572 dv-cinder-849890-4-iscsi devstack@keystone.service[80534]: spawned uWSGI master process (pid: 80534) Aug 05 15:11:20.240780 dv-cinder-849890-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 05 15:11:20.241982 dv-cinder-849890-4-iscsi devstack@keystone.service[80534]: spawned uWSGI worker 1 (pid: 80541, cores: 1) Aug 05 15:11:20.241982 dv-cinder-849890-4-iscsi devstack@keystone.service[80534]: spawned uWSGI worker 2 (pid: 80542, cores: 1) Aug 05 15:11:20.241982 dv-cinder-849890-4-iscsi devstack@keystone.service[80534]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 05 15:11:25.771286 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 05 15:11:25.771286 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: from cryptography.utils import int_from_bytes Aug 05 15:11:25.772109 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 05 15:11:25.772109 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: from cryptography.utils import int_from_bytes Aug 05 15:11:25.772230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 05 15:11:25.772230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: from cryptography.utils import int_from_bytes Aug 05 15:11:25.772637 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 05 15:11:25.772637 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: from cryptography.utils import int_from_bytes Aug 05 15:11:25.821476 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 05 15:11:25.821667 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 05 15:11:25.821865 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] command line args: [] {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 05 15:11:25.822022 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 05 15:11:25.822134 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 05 15:11:25.822212 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 05 15:11:25.822334 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ================================================================================ {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 05 15:11:25.822418 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] command line args: [] {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 05 15:11:25.822594 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] admin_token = **** {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.822674 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 05 15:11:25.822843 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] config_dir = [] {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.822926 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ================================================================================ {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 05 15:11:25.823032 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.823111 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] admin_token = **** {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.823214 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] config_source = [] {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.823328 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] config_dir = [] {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.823406 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] debug = True {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.823519 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.823597 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.823823 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] config_source = [] {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.823892 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] debug = True {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.823955 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.824113 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.824244 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.824339 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.824443 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.824547 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.824654 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.824759 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.824861 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] list_limit = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.824959 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.825053 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] log_config_append = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.825145 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.825242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] list_limit = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.825353 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] log_dir = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.825447 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] log_config_append = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.825540 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] log_file = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.825632 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.825730 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] log_dir = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.825829 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.825951 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] log_file = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.826069 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.826190 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.826309 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.826425 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.826543 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.826668 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.826771 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.826771 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.826989 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.826989 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.827210 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.827313 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.827424 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.827535 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.827651 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.827761 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.827882 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.828002 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.828138 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.828253 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.828369 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.828488 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.828615 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.828728 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.828846 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.828959 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.829069 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.829191 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.829191 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.829414 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.40/identity {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.829526 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] publish_errors = False {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.829673 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.829794 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.40/identity {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.829907 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] publish_errors = False {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.830017 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.830175 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.830288 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.830288 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.830504 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.830614 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.830746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.830870 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.830989 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] standard_threads = False {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.831102 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.831212 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.831323 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] standard_threads = False {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.831441 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.831441 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.831663 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.831775 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] use_journal = False {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.831885 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.832002 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] use_json = False {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.832148 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.832271 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] use_stderr = False {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.832385 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] use_journal = False {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.832496 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] use_syslog = False {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.832607 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] use_json = False {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.832724 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.832847 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] use_stderr = False {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.832847 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] use_syslog = False {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.833063 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.833181 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 15:11:25.833294 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.833410 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.833597 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.833733 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.833847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.833958 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.834069 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.834187 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.834306 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.834419 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.834535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.834653 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.834770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.834893 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.835006 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.835123 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.835234 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.835346 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.835457 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.835575 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.835693 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.835804 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.835915 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.836026 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.836155 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.836282 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.836400 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.836517 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.836636 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.836748 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.836859 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.836977 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.837096 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.837208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.837318 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.837430 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.837540 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.837662 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.837775 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.837891 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.838516 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.838635 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.838746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.838864 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.838983 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.839133 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.839253 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.839371 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.839483 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.839668 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.839782 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.839892 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.840002 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.840155 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.840269 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.840395 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.840510 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.840621 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.840732 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.840843 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.840954 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.841072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.841191 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.841303 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.841421 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.841534 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.841651 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.841773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.841892 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.842003 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.842114 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.842230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.842351 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.842463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.842573 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.842684 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.842794 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.842910 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.843014 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.843109 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.843208 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.843302 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.843396 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.843526 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.843635 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.843731 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.843825 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.843925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.844046 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.844174 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.844297 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.844417 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.844529 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.844640 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.844753 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.844864 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.844988 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.845109 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.845222 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.845332 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.845451 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.845564 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.845666 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.845784 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.845897 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.845999 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.846116 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.846231 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.846344 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.846463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.846583 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.846686 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.846798 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.846915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.847028 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.847147 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.847260 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.847371 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.847483 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.847595 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.847719 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.847840 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.847953 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.848101 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.848231 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.848345 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.848465 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.848465 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.848686 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.848806 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.848920 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.849032 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.849150 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.849150 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.849368 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.849481 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.849599 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.849708 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.849820 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.849932 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.850051 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.850051 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.850262 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.850380 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.850499 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.850612 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.850803 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.850919 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.851031 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.851143 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.851255 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.851367 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.851486 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.851600 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.851717 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.851831 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.851943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.852055 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.852199 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.852325 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.852438 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.852551 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.852668 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.852787 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.852900 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.853020 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.853141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.853141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.853352 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.853352 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.853622 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.854256 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.854371 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.854484 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.854596 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.854708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.854820 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.854941 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.855055 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.855168 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.855281 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.855393 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.855393 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.855618 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.855732 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.855850 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.855969 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.856107 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.856223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.856351 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.856466 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.856579 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.856696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.856810 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.856922 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.857042 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.857156 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.857269 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.857381 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.857492 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.857604 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.857723 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.857837 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.857949 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.858061 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] auth.external = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.858180 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.858299 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.858421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.858535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.858648 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.858760 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.858863 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] auth.external = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.859011 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] auth.password = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.859138 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.859259 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] auth.token = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.859382 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.859497 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.859610 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.859723 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.859841 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] auth.password = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.859963 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.860091 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] auth.token = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.860205 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.860318 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.860433 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.860553 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.860674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.860788 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.860901 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.861014 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.861127 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] credential.caching = True {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.861240 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.861364 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.861480 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.861594 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.861707 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.861820 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.861932 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] credential.caching = True {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.862056 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.862177 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.862292 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.862395 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.862508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.862628 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.862749 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.862867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.862981 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.863093 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.863206 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.863319 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.863439 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.863555 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.863668 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.863780 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.863893 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.864005 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.864171 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.864294 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.864408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.864527 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.864651 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.864766 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.864955 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] federation.caching = True {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.865071 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.865185 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.865297 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.865410 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.865535 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.865658 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.865773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] federation.caching = True {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.865887 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.866000 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.866119 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.866237 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.866359 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.866473 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.866586 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.866700 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.866813 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.866925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.867047 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.867162 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.867275 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.867387 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.867501 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.867613 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.867734 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] identity.caching = True {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.867855 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.867969 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.868099 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.868215 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.868328 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.868453 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.868568 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] identity.caching = True {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.868682 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.868794 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.868908 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.869021 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.869142 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.869294 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.869408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.869521 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.869635 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.869756 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.869873 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.869986 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.870099 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.870212 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.870325 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.870445 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.870560 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.870673 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.870786 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.870900 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.871012 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.871133 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.871248 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.871360 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.871473 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.871586 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.871699 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.871820 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.871935 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.872048 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.872225 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.872338 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.872452 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.872576 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.872693 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.872812 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.872927 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.873041 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.873155 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.873275 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.873398 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.873513 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.873626 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.873739 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.873852 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.873973 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.874090 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.874203 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.874316 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.874429 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.874542 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.874662 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.874778 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.874891 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.875005 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.875119 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.875232 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.875353 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.875468 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.875582 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.875695 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.876055 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.876183 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.876183 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.876414 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.876533 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.876647 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.876760 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.876862 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.876862 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.877023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.877101 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.877172 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.877241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.877312 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.877389 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.877461 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.877531 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.877602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.877672 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.877742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.877819 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.877892 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.877961 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.878032 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.878108 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.878180 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.878258 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.878331 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.878401 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.878471 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.878620 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.878699 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.878871 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.user = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.878950 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.879107 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.879185 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.879348 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.879426 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.879582 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.879661 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.879814 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.user = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.879892 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.880057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.880176 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.880327 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.880406 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.880562 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.880640 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.880818 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.880903 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.881030 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.881110 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.881264 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.881343 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.881500 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.881592 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.881726 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.881804 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.881968 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.882046 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.882207 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.882293 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.882445 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.882524 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.882677 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.882790 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.882926 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.883041 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.883175 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.883275 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.883423 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.883523 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.883673 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.883762 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.883923 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.884001 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.884181 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.884262 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.884433 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.884523 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.884673 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.884752 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.884938 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.885018 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.885178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.885264 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.885432 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.885509 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.885665 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.885743 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.885910 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.885997 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.886147 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.886225 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.886383 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.886460 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.886621 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.886775 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.886871 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.886949 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.887113 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.887191 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.887346 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.887431 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.887583 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.887661 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.887823 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] resource.caching = True {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.887907 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.888104 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.888196 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.888325 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.888403 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.888559 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.888635 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.888799 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] resource.caching = True {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.888884 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.889047 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.889124 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.889286 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.889366 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.889520 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.889604 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.889754 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.889832 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.889993 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.890070 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.890228 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] role.caching = True {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.890313 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.890474 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.890551 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] role.driver = sql {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.890702 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.890779 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.890942 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.891026 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.891183 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] role.caching = True {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.891262 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.891428 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.891506 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] role.driver = sql {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.891659 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.891743 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.891918 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.891997 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.892187 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.892266 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.892437 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.892526 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.892683 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.892766 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.892916 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.892995 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.893165 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.893259 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.893408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.893491 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.893655 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.893734 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.893887 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.893966 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.894139 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.894288 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.894390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.894525 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.894642 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.894776 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.894893 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.895024 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.895274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.895351 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.895523 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.895602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.895756 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.895833 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.896008 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.896105 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.896289 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.896370 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.896526 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.896604 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.896759 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.896846 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.896992 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.897072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.897226 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.897304 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.897477 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.897553 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.897713 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.897794 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.897949 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.898028 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.898183 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.898261 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.898421 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.898501 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.898654 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.898733 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.898891 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] token.caching = True {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.898968 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.899127 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.899212 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.899363 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.899440 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.899597 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.899682 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.899834 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] token.caching = True {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.899911 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.900123 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.900207 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.900331 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.900919 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.901085 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.901163 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.901327 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.901404 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.901576 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.901662 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.901812 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.901889 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.902045 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.902122 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.902280 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.902367 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.902525 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.902604 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.902760 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.902836 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.902996 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.903080 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.903231 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.903308 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.903511 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.903597 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.903718 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.903845 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] database.connection = **** {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.903972 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.904124 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.904241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.904355 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.904470 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.904583 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.904696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.904817 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] database.connection = **** {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.904943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.905066 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.905181 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.905294 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.905407 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.905521 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.905672 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.905788 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.905902 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.906015 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.906128 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.906242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.906430 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.906548 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.906661 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.906774 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.906887 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.907001 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.907126 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.907241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.907355 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.907468 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.907582 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.907696 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.907817 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.907931 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.908046 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.908189 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.908303 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 05 15:11:25.908417 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.908417 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.908644 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.908798 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 15:11:25.908970 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80541) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 05 15:11:25.937343 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:25.937575 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:25.937802 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:25.937886 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:25.938117 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:25.938346 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:25.946076 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:25.946671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:25.975767 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:25.975992 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:25.976704 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:25.976954 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.030035 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.030269 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.080562 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.080792 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.081825 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.082062 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.145201 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.145429 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.145654 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.145768 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.145875 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.146004 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.146087 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.146233 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.146314 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.146457 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.146679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.146894 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.169515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.169721 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.169854 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.169961 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.170059 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.170158 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.170256 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.170354 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.170461 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.170561 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.170672 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.170770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.170868 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.170973 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.182572 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.182773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.195136 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.195367 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.195580 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.195664 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.195797 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.195877 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.196013 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.196137 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.196254 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.196336 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.196470 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.196556 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.196682 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.196896 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80541) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 15:11:26.213127 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.213372 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 05 15:11:26.213633 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.213716 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.213852 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 05 15:11:26.213935 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 05 15:11:26.214147 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.214227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.214380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 05 15:11:26.214460 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.214652 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.214734 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.214901 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.214981 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.215154 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.215411 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.224548 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.224827 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.225092 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.225257 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.225373 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.225537 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.225807 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.226090 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.232805 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 15:11:26.233678 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 15:11:26.237454 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 15:11:26.237730 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.238003 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.238276 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.238409 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 15:11:26.238550 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.238690 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.238805 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.238972 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.239053 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.239254 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.239343 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.239538 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.239628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.239797 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.239879 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.240084 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.240169 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.240363 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.240633 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.240904 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.241168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.263636 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.265144 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 15:11:26.265332 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.265416 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.266821 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 15:11:26.267105 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.269976 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 15:11:26.270255 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.270535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.271725 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 15:11:26.272006 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.272310 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.277159 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 15:11:26.277439 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.279111 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 15:11:26.279407 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.281574 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 15:11:26.281818 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.282095 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.282377 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.282664 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.282940 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.283211 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.283482 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.283561 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 15:11:26.283787 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.284091 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.284391 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.284691 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.284991 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.285261 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.285540 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.299757 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.300030 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.301981 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.302263 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.303959 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 15:11:26.306217 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 15:11:26.307683 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.307973 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.309934 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.310234 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.312379 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 15:11:26.314652 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 15:11:26.315522 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 15:11:26.317914 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 15:11:26.318689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.318986 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.319269 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.319564 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.319849 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.320150 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.321461 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.321773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.322062 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.322388 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.322677 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.322965 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.336093 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 15:11:26.336380 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.336644 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.336911 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.338974 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 15:11:26.339271 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.339700 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.339829 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.343765 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.345341 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.345623 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.346716 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.348304 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.348446 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 15:11:26.348592 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.348726 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.349017 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.349292 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.349561 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.351422 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 15:11:26.351705 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.352027 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.352353 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.352633 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.361497 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 15:11:26.361774 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.362050 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.362326 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.362604 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.362875 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.363157 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.364907 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 15:11:26.365186 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.365462 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.365742 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.366021 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.366304 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.366587 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.379547 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 15:11:26.382528 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 15:11:26.383153 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 15:11:26.385520 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.386210 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 15:11:26.388007 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.389172 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.389545 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 15:11:26.389839 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.390126 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.391618 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.393150 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 15:11:26.393442 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.393747 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.396559 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.398154 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 15:11:26.400261 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.401144 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.401430 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.401711 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.401907 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 15:11:26.401990 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.404896 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.405186 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.405468 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.405747 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.409998 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 15:11:26.410295 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.410574 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.413815 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 15:11:26.414117 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.414398 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.417955 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 15:11:26.418237 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.418516 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.418788 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.419057 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.419328 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.419599 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.419881 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.420276 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.420559 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.420837 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.421099 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.421364 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.421631 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.421858 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80541) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 15:11:26.422140 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.422420 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.422695 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.422965 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.423238 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.423512 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.423795 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.424199 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.424485 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.424767 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.425029 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.425295 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.425573 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80541) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 15:11:26.455279 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d197d249c0 pid: 80542 (default app) Aug 05 15:11:26.458587 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8199ad1-4100-4dde-a912-b6c7985374f0 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:26.458881 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8199ad1-4100-4dde-a912-b6c7985374f0 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:26.459159 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8199ad1-4100-4dde-a912-b6c7985374f0 None None] PATH_INFO: `/v3` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:26.459390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d197d249c0 pid: 80541 (default app) Aug 05 15:11:26.463868 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1/1] 10.222.0.40 () {50 vars in 791 bytes} [Fri Aug 5 15:11:26 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:11:34.982271 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879d303-acea-44e2-a1c8-e611fd95e9c8 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:34.982785 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879d303-acea-44e2-a1c8-e611fd95e9c8 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:34.982867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879d303-acea-44e2-a1c8-e611fd95e9c8 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:34.987236 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1/2] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:11:34 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 15:11:34.992599 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523944d5-a095-40c9-9e1d-ad212ee495d4 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:34.992997 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523944d5-a095-40c9-9e1d-ad212ee495d4 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:34.993152 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523944d5-a095-40c9-9e1d-ad212ee495d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:35.065818 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: INFO dbcounter [None req-523944d5-a095-40c9-9e1d-ad212ee495d4 None None] Registered counter for database keystone Aug 05 15:11:35.084642 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG oslo_db.sqlalchemy.engines [None req-523944d5-a095-40c9-9e1d-ad212ee495d4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80542) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 05 15:11:35.091479 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writer thread running {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 05 15:11:35.268840 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG passlib.handlers.bcrypt [None req-523944d5-a095-40c9-9e1d-ad212ee495d4 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80542) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 05 15:11:35.269264 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG passlib.handlers.bcrypt [None req-523944d5-a095-40c9-9e1d-ad212ee495d4 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80542) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 05 15:11:35.323566 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-523944d5-a095-40c9-9e1d-ad212ee495d4 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:11:35.325226 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-523944d5-a095-40c9-9e1d-ad212ee495d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:11:35.454782 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 2/3] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:11:34 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 05 15:11:35.588686 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a879d303-acea-44e2-a1c8-e611fd95e9c8 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:35.666111 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: INFO dbcounter [None req-a879d303-acea-44e2-a1c8-e611fd95e9c8 None None] Registered counter for database keystone Aug 05 15:11:35.687569 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG oslo_db.sqlalchemy.engines [None req-a879d303-acea-44e2-a1c8-e611fd95e9c8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80541) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 05 15:11:35.693797 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writer thread running {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 05 15:11:35.812964 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a879d303-acea-44e2-a1c8-e611fd95e9c8 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:35.817222 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e2570-097e-4b13-a5ba-63a1ab08e152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:35.817866 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e2570-097e-4b13-a5ba-63a1ab08e152 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:35.818154 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e2570-097e-4b13-a5ba-63a1ab08e152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:35.818456 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e2570-097e-4b13-a5ba-63a1ab08e152 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:35.827064 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200e2570-097e-4b13-a5ba-63a1ab08e152 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:35.878109 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: WARNING keystone.common.rbac_enforcer.enforcer [None req-200e2570-097e-4b13-a5ba-63a1ab08e152 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 05 15:11:35.878396 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200e2570-097e-4b13-a5ba-63a1ab08e152 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:35.888579 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 2/4] 10.222.0.40 () {58 vars in 1324 bytes} [Fri Aug 5 15:11:35 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 15:11:35.899652 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523944d5-a095-40c9-9e1d-ad212ee495d4 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:35.919351 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523944d5-a095-40c9-9e1d-ad212ee495d4 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:35.925954 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a75fba-6d2b-4908-ae8e-980ce771e4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:35.927646 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a75fba-6d2b-4908-ae8e-980ce771e4ec admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:35.928125 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a75fba-6d2b-4908-ae8e-980ce771e4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:35.928558 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a75fba-6d2b-4908-ae8e-980ce771e4ec admin admin] PATH_INFO: `/v3/projects/0744d3ce240a4f198b94f2d1e6d62c8e` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:35.930842 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a75fba-6d2b-4908-ae8e-980ce771e4ec admin admin] RBAC: Authorizing `identity:get_project(project_id=0744d3ce240a4f198b94f2d1e6d62c8e)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:35.988662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: WARNING keystone.common.rbac_enforcer.enforcer [None req-25a75fba-6d2b-4908-ae8e-980ce771e4ec admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 05 15:11:35.989013 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a75fba-6d2b-4908-ae8e-980ce771e4ec admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:35.994444 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 3/5] 10.222.0.40 () {56 vars in 1207 bytes} [Fri Aug 5 15:11:35 2022] GET /identity/v3/projects/0744d3ce240a4f198b94f2d1e6d62c8e => generated 344 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:11:37.971360 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41cb2e-18d1-41cd-b853-71a9e6fc5986 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:37.971934 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41cb2e-18d1-41cd-b853-71a9e6fc5986 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:37.971934 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41cb2e-18d1-41cd-b853-71a9e6fc5986 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:37.976132 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 3/6] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:11:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 15:11:37.982114 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a61241-17ca-45dd-9508-df497194b71e None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:37.982558 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a61241-17ca-45dd-9508-df497194b71e None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:37.982651 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a61241-17ca-45dd-9508-df497194b71e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:38.002409 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-42a61241-17ca-45dd-9508-df497194b71e None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:11:38.003934 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-42a61241-17ca-45dd-9508-df497194b71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:11:38.011748 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 4/7] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:11:37 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 05 15:11:38.033508 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb41cb2e-18d1-41cd-b853-71a9e6fc5986 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:38.046109 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb41cb2e-18d1-41cd-b853-71a9e6fc5986 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:38.050334 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b92298-64d9-4383-91dc-b8a386cb456e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:38.050936 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b92298-64d9-4383-91dc-b8a386cb456e admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:38.051225 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b92298-64d9-4383-91dc-b8a386cb456e admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:38.051505 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b92298-64d9-4383-91dc-b8a386cb456e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:38.059146 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b92298-64d9-4383-91dc-b8a386cb456e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:38.060954 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b92298-64d9-4383-91dc-b8a386cb456e admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:38.069655 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 4/8] 10.222.0.40 () {58 vars in 1324 bytes} [Fri Aug 5 15:11:38 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 15:11:38.074603 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a61241-17ca-45dd-9508-df497194b71e None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:38.083367 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a61241-17ca-45dd-9508-df497194b71e None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:38.088249 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70a1614-d69d-4cb3-96ad-379d369cf5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:38.088866 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a1614-d69d-4cb3-96ad-379d369cf5ee admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:38.089150 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a1614-d69d-4cb3-96ad-379d369cf5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:38.089427 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a1614-d69d-4cb3-96ad-379d369cf5ee admin admin] PATH_INFO: `/v3/users/338271833df8486b9ef50964b5ec7797` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:38.090776 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70a1614-d69d-4cb3-96ad-379d369cf5ee admin admin] RBAC: Authorizing `identity:get_user(user_id=338271833df8486b9ef50964b5ec7797)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:38.092612 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70a1614-d69d-4cb3-96ad-379d369cf5ee admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:38.094126 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 5/9] 10.222.0.40 () {56 vars in 1198 bytes} [Fri Aug 5 15:11:38 2022] GET /identity/v3/users/338271833df8486b9ef50964b5ec7797 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:11:40.129253 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8cac7-8d40-4d89-980f-8a99cd4c3de5 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:40.129789 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8cac7-8d40-4d89-980f-8a99cd4c3de5 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:40.129789 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8cac7-8d40-4d89-980f-8a99cd4c3de5 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:40.131137 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 5/10] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:11:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:11:40.136791 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a94b3-461f-47e9-8b3d-e3c7d7ffbf47 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:40.137212 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a94b3-461f-47e9-8b3d-e3c7d7ffbf47 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:40.137351 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a94b3-461f-47e9-8b3d-e3c7d7ffbf47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:40.156150 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-3f1a94b3-461f-47e9-8b3d-e3c7d7ffbf47 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:11:40.157334 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-3f1a94b3-461f-47e9-8b3d-e3c7d7ffbf47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:11:40.171638 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a108ce0-8f6c-4edb-954c-059d667cb070 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:40.171925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a108ce0-8f6c-4edb-954c-059d667cb070 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:40.172229 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a108ce0-8f6c-4edb-954c-059d667cb070 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:40.173587 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 6/11] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:11:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 15:11:40.178169 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674f10a-7e35-4a92-9e5a-9564e34b88e7 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:40.178452 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674f10a-7e35-4a92-9e5a-9564e34b88e7 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:40.178735 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674f10a-7e35-4a92-9e5a-9564e34b88e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:40.220725 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG passlib.handlers.bcrypt [None req-b674f10a-7e35-4a92-9e5a-9564e34b88e7 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80541) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 05 15:11:40.221160 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG passlib.handlers.bcrypt [None req-b674f10a-7e35-4a92-9e5a-9564e34b88e7 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80541) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 05 15:11:40.239515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 6/12] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:11:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:11:40.263792 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1a94b3-461f-47e9-8b3d-e3c7d7ffbf47 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:40.267727 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-b674f10a-7e35-4a92-9e5a-9564e34b88e7 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:11:40.269351 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-b674f10a-7e35-4a92-9e5a-9564e34b88e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:11:40.286459 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:40.364815 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1a94b3-461f-47e9-8b3d-e3c7d7ffbf47 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:40.369453 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 7/13] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:11:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 15:11:40.391940 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674f10a-7e35-4a92-9e5a-9564e34b88e7 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:40.399321 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a9e83-d62d-4c8b-b794-faa8fb7c10cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:40.400004 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a9e83-d62d-4c8b-b794-faa8fb7c10cc None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:40.400321 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a9e83-d62d-4c8b-b794-faa8fb7c10cc None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:40.400604 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a9e83-d62d-4c8b-b794-faa8fb7c10cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:40.412165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a9e83-d62d-4c8b-b794-faa8fb7c10cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:11:40.421274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-012a9e83-d62d-4c8b-b794-faa8fb7c10cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:11:40.421274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:11:40.421274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:11:40.421274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:11:40.421274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:11:40.421274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:11:40.421274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:11:40.421274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:11:40.421274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:11:40.421274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:11:40.421274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:11:40.421274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:11:40.421274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:11:40.421274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:11:40.421274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:11:40.421274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:11:40.422206 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:11:40.422206 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:11:40.422206 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:11:40.422206 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:11:40.422206 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:11:40.422206 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:11:40.422206 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:11:40.422206 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:11:40.422206 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:11:40.422206 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:11:40.422206 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:11:40.422206 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:11:40.422206 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:11:40.422206 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:11:40.422206 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:11:40.422206 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:11:40.423023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:11:40.423023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:11:40.423023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:11:40.423023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:11:40.423023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:11:40.423023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:11:40.423023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:11:40.423023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:11:40.423023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:11:40.423023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:11:40.423023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:11:40.423023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:11:40.423023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 7/14] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 15:11:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:11:40.428533 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a9e83-d62d-4c8b-b794-faa8fb7c10cc None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:40.493147 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674f10a-7e35-4a92-9e5a-9564e34b88e7 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:40.495958 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:40.512392 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:40.530912 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a9e83-d62d-4c8b-b794-faa8fb7c10cc None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:40.532593 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89191f5-27ac-42fa-802b-0b1b6d470f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:40.533530 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89191f5-27ac-42fa-802b-0b1b6d470f9a None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:40.533819 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89191f5-27ac-42fa-802b-0b1b6d470f9a None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:40.534094 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89191f5-27ac-42fa-802b-0b1b6d470f9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:40.564905 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292d68a4-49ef-42b0-b017-296e6631c59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:40.565518 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d68a4-49ef-42b0-b017-296e6631c59f None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:40.565801 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d68a4-49ef-42b0-b017-296e6631c59f None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:40.566077 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d68a4-49ef-42b0-b017-296e6631c59f None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:40.566884 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292d68a4-49ef-42b0-b017-296e6631c59f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:11:40.567391 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292d68a4-49ef-42b0-b017-296e6631c59f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:40.569519 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292d68a4-49ef-42b0-b017-296e6631c59f None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:40.581285 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 8/15] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:11:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:11:40.586837 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292d68a4-49ef-42b0-b017-296e6631c59f None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:40.604040 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89191f5-27ac-42fa-802b-0b1b6d470f9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:40.605666 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89191f5-27ac-42fa-802b-0b1b6d470f9a None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:40.683970 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292d68a4-49ef-42b0-b017-296e6631c59f None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:40.703058 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 8/16] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:11:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 15:11:40.708667 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89191f5-27ac-42fa-802b-0b1b6d470f9a None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:40.711337 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:40.719021 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d776d2ad-d2be-4125-b1ee-48e6b2975019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:40.725303 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776d2ad-d2be-4125-b1ee-48e6b2975019 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:40.725303 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776d2ad-d2be-4125-b1ee-48e6b2975019 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:40.725303 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776d2ad-d2be-4125-b1ee-48e6b2975019 None admin] PATH_INFO: `/v3/users/338271833df8486b9ef50964b5ec7797` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:40.725303 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d776d2ad-d2be-4125-b1ee-48e6b2975019 None admin] RBAC: Authorizing `identity:get_user(user_id=338271833df8486b9ef50964b5ec7797)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:40.725303 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d776d2ad-d2be-4125-b1ee-48e6b2975019 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:40.725303 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 9/17] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:11:40 2022] GET /identity/v3/users/338271833df8486b9ef50964b5ec7797 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:11:40.730384 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d776d2ad-d2be-4125-b1ee-48e6b2975019 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:40.745114 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:40.811564 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89191f5-27ac-42fa-802b-0b1b6d470f9a None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:40.849983 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8f6746-3efd-48e3-95d7-2538a6083633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:40.850872 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f6746-3efd-48e3-95d7-2538a6083633 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:40.851162 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f6746-3efd-48e3-95d7-2538a6083633 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:40.851445 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f6746-3efd-48e3-95d7-2538a6083633 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:40.871807 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8f6746-3efd-48e3-95d7-2538a6083633 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:40.873343 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8f6746-3efd-48e3-95d7-2538a6083633 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:40.892998 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-da8f6746-3efd-48e3-95d7-2538a6083633 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:11:40.892998 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:11:40.892998 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:11:40.892998 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:11:40.892998 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:11:40.892998 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:11:40.892998 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:11:40.892998 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:11:40.892998 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:11:40.892998 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:11:40.892998 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:11:40.892998 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:11:40.892998 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:11:40.892998 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:11:40.892998 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:11:40.893930 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:11:40.893930 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:11:40.893930 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:11:40.893930 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:11:40.893930 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:11:40.893930 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:11:40.893930 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:11:40.893930 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:11:40.893930 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:11:40.893930 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:11:40.893930 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:11:40.893930 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:11:40.893930 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:11:40.893930 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:11:40.893930 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:11:40.893930 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:11:40.893930 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:11:40.894714 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:11:40.894714 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:11:40.894714 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:11:40.894714 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:11:40.894714 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 9/18] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:11:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:11:40.900441 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8f6746-3efd-48e3-95d7-2538a6083633 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:40.937349 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d776d2ad-d2be-4125-b1ee-48e6b2975019 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:40.954499 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:40.970402 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c15119-e4f9-4cd4-b515-a1252ecf47f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:40.971046 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c15119-e4f9-4cd4-b515-a1252ecf47f2 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:40.971331 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c15119-e4f9-4cd4-b515-a1252ecf47f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:40.971610 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c15119-e4f9-4cd4-b515-a1252ecf47f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:40.972833 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c15119-e4f9-4cd4-b515-a1252ecf47f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:40.974296 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c15119-e4f9-4cd4-b515-a1252ecf47f2 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:40.975870 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 10/19] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:11:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:11:40.982057 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c15119-e4f9-4cd4-b515-a1252ecf47f2 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:40.999184 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8f6746-3efd-48e3-95d7-2538a6083633 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:41.032839 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd0a003-5f48-4204-9ae1-643160b62cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:41.033454 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0a003-5f48-4204-9ae1-643160b62cc2 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:41.033744 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0a003-5f48-4204-9ae1-643160b62cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:41.034026 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0a003-5f48-4204-9ae1-643160b62cc2 None admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:41.034719 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd0a003-5f48-4204-9ae1-643160b62cc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:11:41.035028 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd0a003-5f48-4204-9ae1-643160b62cc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:41.036437 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd0a003-5f48-4204-9ae1-643160b62cc2 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:41.054212 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 10/20] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:11:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:11:41.063056 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:41.083737 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c15119-e4f9-4cd4-b515-a1252ecf47f2 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:41.121999 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7b88fd-7a93-48c2-aa46-c174257f607c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:41.122638 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b88fd-7a93-48c2-aa46-c174257f607c None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:41.122926 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b88fd-7a93-48c2-aa46-c174257f607c None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:41.123207 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b88fd-7a93-48c2-aa46-c174257f607c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:41.124016 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7b88fd-7a93-48c2-aa46-c174257f607c None admin] RBAC: Adding query filter params (role.id=7c88c815db854954b516ab97cd471647, scope.domain.id=default, user.id=338271833df8486b9ef50964b5ec7797) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:11:41.124365 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7b88fd-7a93-48c2-aa46-c174257f607c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:41.126115 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7b88fd-7a93-48c2-aa46-c174257f607c None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:41.133738 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 11/21] 10.222.0.40 () {56 vars in 1322 bytes} [Fri Aug 5 15:11:40 2022] GET /identity/v3/role_assignments?user.id=338271833df8486b9ef50964b5ec7797&scope.domain.id=default&role.id=7c88c815db854954b516ab97cd471647 => generated 233 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:11:43.475446 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0955c-4fc5-4d7d-8e57-b37f6aad96e1 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:43.475926 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0955c-4fc5-4d7d-8e57-b37f6aad96e1 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:43.475926 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0955c-4fc5-4d7d-8e57-b37f6aad96e1 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:43.477824 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 11/22] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:11:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:11:43.483298 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49dc271-b085-47b8-ab63-ee471a3f6a4c None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:43.484260 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49dc271-b085-47b8-ab63-ee471a3f6a4c None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:43.484743 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49dc271-b085-47b8-ab63-ee471a3f6a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:43.504354 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-f49dc271-b085-47b8-ab63-ee471a3f6a4c None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:11:43.505899 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-f49dc271-b085-47b8-ab63-ee471a3f6a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:11:43.598330 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 12/23] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:11:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:11:43.645612 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0955c-4fc5-4d7d-8e57-b37f6aad96e1 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:43.656751 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2e464-99cd-42d8-a720-3270dc0a129c None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:43.659576 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2e464-99cd-42d8-a720-3270dc0a129c None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:43.660112 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2e464-99cd-42d8-a720-3270dc0a129c None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:43.661683 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 13/24] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:11:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:11:43.667978 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa7d73-02a6-4f27-af0c-4c65f715b1de None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:43.668777 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa7d73-02a6-4f27-af0c-4c65f715b1de None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:43.670728 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa7d73-02a6-4f27-af0c-4c65f715b1de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:43.701514 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-b1fa7d73-02a6-4f27-af0c-4c65f715b1de None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:11:43.705002 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-b1fa7d73-02a6-4f27-af0c-4c65f715b1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:11:43.831344 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0955c-4fc5-4d7d-8e57-b37f6aad96e1 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:43.847119 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 14/25] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:11:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:11:43.876483 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:43.883549 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1344a21-464c-4711-8f6a-b252ca39ba53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:43.884296 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1344a21-464c-4711-8f6a-b252ca39ba53 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:43.884586 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1344a21-464c-4711-8f6a-b252ca39ba53 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:43.885848 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa7d73-02a6-4f27-af0c-4c65f715b1de None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:43.887860 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1344a21-464c-4711-8f6a-b252ca39ba53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:43.889892 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:43.917979 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1344a21-464c-4711-8f6a-b252ca39ba53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:43.919485 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1344a21-464c-4711-8f6a-b252ca39ba53 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:43.955963 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-c1344a21-464c-4711-8f6a-b252ca39ba53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:11:43.955963 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:11:43.955963 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:11:43.955963 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:11:43.955963 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:11:43.955963 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:11:43.955963 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:11:43.955963 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:11:43.955963 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:11:43.955963 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:11:43.955963 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:11:43.955963 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:11:43.955963 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:11:43.955963 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:11:43.955963 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:11:43.956878 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:11:43.956878 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:11:43.956878 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:11:43.956878 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:11:43.956878 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:11:43.956878 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:11:43.956878 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:11:43.956878 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:11:43.956878 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:11:43.956878 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:11:43.956878 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:11:43.956878 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:11:43.956878 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:11:43.956878 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:11:43.956878 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:11:43.956878 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:11:43.956878 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:11:43.957675 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:11:43.957675 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:11:43.957675 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:11:43.957675 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:11:43.968695 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 12/26] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:11:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:11:43.978996 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a033063-1365-4534-9858-dd8fdf8abd00 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:43.979283 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a033063-1365-4534-9858-dd8fdf8abd00 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:43.979567 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a033063-1365-4534-9858-dd8fdf8abd00 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:43.988473 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 13/27] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:11:43 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:11:43.989268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a033063-1365-4534-9858-dd8fdf8abd00 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:44.064736 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa7d73-02a6-4f27-af0c-4c65f715b1de None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:44.103274 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a033063-1365-4534-9858-dd8fdf8abd00 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:44.103562 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6fad91-d2f7-47da-ac40-c6418bfc81fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:44.103861 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6fad91-d2f7-47da-ac40-c6418bfc81fa None admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:44.104181 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6fad91-d2f7-47da-ac40-c6418bfc81fa None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:44.104465 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6fad91-d2f7-47da-ac40-c6418bfc81fa None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:44.105294 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6fad91-d2f7-47da-ac40-c6418bfc81fa None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:44.106823 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6fad91-d2f7-47da-ac40-c6418bfc81fa None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:44.140705 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a7b3f-15fa-46b3-83d7-0c44ed3e5309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:44.141329 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a7b3f-15fa-46b3-83d7-0c44ed3e5309 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:44.141614 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a7b3f-15fa-46b3-83d7-0c44ed3e5309 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:44.141893 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a7b3f-15fa-46b3-83d7-0c44ed3e5309 None admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:44.142625 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a7b3f-15fa-46b3-83d7-0c44ed3e5309 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:11:44.142906 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a7b3f-15fa-46b3-83d7-0c44ed3e5309 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:44.144305 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a7b3f-15fa-46b3-83d7-0c44ed3e5309 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:44.160104 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 14/28] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:11:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:11:44.160104 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0036bb89-4548-45bd-b168-ebcfc068ae2e None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:44.160104 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0036bb89-4548-45bd-b168-ebcfc068ae2e None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:44.160104 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0036bb89-4548-45bd-b168-ebcfc068ae2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:44.288817 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-0036bb89-4548-45bd-b168-ebcfc068ae2e None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:11:44.290076 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-0036bb89-4548-45bd-b168-ebcfc068ae2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:11:44.305915 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 15/29] 10.222.0.40 () {60 vars in 1131 bytes} [Fri Aug 5 15:11:43 2022] POST /identity/v3/roles => generated 219 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:11:44.308123 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61f0e5-5df0-4fe0-b121-f8c0181efd11 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:44.308725 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61f0e5-5df0-4fe0-b121-f8c0181efd11 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:44.309416 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61f0e5-5df0-4fe0-b121-f8c0181efd11 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:44.311059 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 16/30] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:11:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:11:44.312209 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61f0e5-5df0-4fe0-b121-f8c0181efd11 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:44.321587 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:44.372878 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 15/31] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:11:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:11:44.380874 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51021e9a-1f1c-42e4-8825-8809049ed3d4 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:44.381195 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51021e9a-1f1c-42e4-8825-8809049ed3d4 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:44.381448 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51021e9a-1f1c-42e4-8825-8809049ed3d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:44.397302 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:44.413234 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-51021e9a-1f1c-42e4-8825-8809049ed3d4 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:11:44.415463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-51021e9a-1f1c-42e4-8825-8809049ed3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:11:44.444088 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61f0e5-5df0-4fe0-b121-f8c0181efd11 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:44.485455 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb01f9b4-6664-45cc-b441-fbd9b863384a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:44.486970 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb01f9b4-6664-45cc-b441-fbd9b863384a None admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:44.487510 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb01f9b4-6664-45cc-b441-fbd9b863384a None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:44.488018 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb01f9b4-6664-45cc-b441-fbd9b863384a None admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:44.488988 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb01f9b4-6664-45cc-b441-fbd9b863384a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:44.491038 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb01f9b4-6664-45cc-b441-fbd9b863384a None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:44.521101 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 16/32] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:11:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:11:44.523120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51021e9a-1f1c-42e4-8825-8809049ed3d4 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:44.615829 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 17/33] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 5 15:11:44 2022] POST /identity/v3/projects => generated 301 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:11:44.617363 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb01f9b4-6664-45cc-b441-fbd9b863384a None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:44.641223 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:44.661911 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51021e9a-1f1c-42e4-8825-8809049ed3d4 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:44.719630 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ecbb14-48fa-491d-9d73-a13b6d38dea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:44.720344 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ecbb14-48fa-491d-9d73-a13b6d38dea8 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:44.720687 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ecbb14-48fa-491d-9d73-a13b6d38dea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:44.721105 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ecbb14-48fa-491d-9d73-a13b6d38dea8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:44.745055 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ecbb14-48fa-491d-9d73-a13b6d38dea8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:11:44.753178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-f9ecbb14-48fa-491d-9d73-a13b6d38dea8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:11:44.753178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:11:44.753178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:11:44.753178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:11:44.753178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:11:44.753178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:11:44.753178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:11:44.753178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:11:44.753178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:11:44.753178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:11:44.753178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:11:44.753178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:11:44.753178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:11:44.753178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:11:44.753178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:11:44.753178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:11:44.754124 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:11:44.754124 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:11:44.754124 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:11:44.754124 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:11:44.754124 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:11:44.754124 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:11:44.754124 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:11:44.754124 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:11:44.754124 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:11:44.754124 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:11:44.754124 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:11:44.754124 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:11:44.754124 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:11:44.754124 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:11:44.754124 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:11:44.754124 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:11:44.754878 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:11:44.754878 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:11:44.754878 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:11:44.754878 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:11:44.754878 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:11:44.754878 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:11:44.754878 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:11:44.754878 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:11:44.754878 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:11:44.754878 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:11:44.754878 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:11:44.754878 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:11:44.755969 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 17/34] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 15:11:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:11:44.756607 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb455bf-c5fd-4f71-9e1f-934293388099 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:44.756906 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb455bf-c5fd-4f71-9e1f-934293388099 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:44.757190 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb455bf-c5fd-4f71-9e1f-934293388099 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:44.759225 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 18/35] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:11:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:11:44.769792 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c915ecb-cf60-4867-a49a-318c3e3db04e None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:44.770078 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c915ecb-cf60-4867-a49a-318c3e3db04e None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:44.772844 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c915ecb-cf60-4867-a49a-318c3e3db04e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:44.781645 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb01f9b4-6664-45cc-b441-fbd9b863384a None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:44.796672 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-0c915ecb-cf60-4867-a49a-318c3e3db04e None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:11:44.798200 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-0c915ecb-cf60-4867-a49a-318c3e3db04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:11:44.803528 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:44.836276 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f12cd6-e9d0-4833-9d4b-116e7ef377b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:44.836968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f12cd6-e9d0-4833-9d4b-116e7ef377b9 None admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:44.837261 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f12cd6-e9d0-4833-9d4b-116e7ef377b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:44.837544 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f12cd6-e9d0-4833-9d4b-116e7ef377b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:44.838301 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f12cd6-e9d0-4833-9d4b-116e7ef377b9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:44.839726 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f12cd6-e9d0-4833-9d4b-116e7ef377b9 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:44.891679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 19/36] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:11:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:11:44.893078 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c915ecb-cf60-4867-a49a-318c3e3db04e None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:44.936096 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 18/37] 10.222.0.40 () {60 vars in 1131 bytes} [Fri Aug 5 15:11:44 2022] POST /identity/v3/roles => generated 225 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:11:44.936096 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f12cd6-e9d0-4833-9d4b-116e7ef377b9 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:45.001269 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:45.031078 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c915ecb-cf60-4867-a49a-318c3e3db04e None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:45.062922 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f12cd6-e9d0-4833-9d4b-116e7ef377b9 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:45.079812 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43424772-b08e-401a-9c99-9f1b85f82158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:45.081030 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43424772-b08e-401a-9c99-9f1b85f82158 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:45.081575 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43424772-b08e-401a-9c99-9f1b85f82158 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:45.081877 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43424772-b08e-401a-9c99-9f1b85f82158 None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:45.082624 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43424772-b08e-401a-9c99-9f1b85f82158 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:11:45.083197 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43424772-b08e-401a-9c99-9f1b85f82158 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:45.085075 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43424772-b08e-401a-9c99-9f1b85f82158 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:45.103817 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 20/38] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:11:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:11:45.108487 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f311be47-1a32-41bc-a65b-f5bde5c4ab20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:45.109357 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311be47-1a32-41bc-a65b-f5bde5c4ab20 None admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:45.109651 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311be47-1a32-41bc-a65b-f5bde5c4ab20 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:45.109939 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311be47-1a32-41bc-a65b-f5bde5c4ab20 None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:45.111430 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f311be47-1a32-41bc-a65b-f5bde5c4ab20 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:45.112100 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43424772-b08e-401a-9c99-9f1b85f82158 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:45.113006 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f311be47-1a32-41bc-a65b-f5bde5c4ab20 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:45.182790 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 19/39] 10.222.0.40 () {60 vars in 1131 bytes} [Fri Aug 5 15:11:44 2022] POST /identity/v3/roles => generated 223 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:11:45.216464 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc0670-2c09-435a-a2de-865d8246dad1 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:45.216789 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc0670-2c09-435a-a2de-865d8246dad1 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:45.217078 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc0670-2c09-435a-a2de-865d8246dad1 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:45.218637 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 20/40] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:11:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 15:11:45.224184 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a9707-7d69-42a0-8872-ad147a75697b None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:45.224486 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a9707-7d69-42a0-8872-ad147a75697b None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:45.224775 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a9707-7d69-42a0-8872-ad147a75697b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:45.237683 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43424772-b08e-401a-9c99-9f1b85f82158 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:45.247126 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-411a9707-7d69-42a0-8872-ad147a75697b None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:11:45.248783 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-411a9707-7d69-42a0-8872-ad147a75697b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:11:45.282051 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:45.287812 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995d4fe-64c5-4d7a-aec6-a237ab18bc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:45.288531 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995d4fe-64c5-4d7a-aec6-a237ab18bc57 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:45.294882 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995d4fe-64c5-4d7a-aec6-a237ab18bc57 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:45.294882 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995d4fe-64c5-4d7a-aec6-a237ab18bc57 None admin] PATH_INFO: `/v3/users/338271833df8486b9ef50964b5ec7797` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:45.294882 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3995d4fe-64c5-4d7a-aec6-a237ab18bc57 None admin] RBAC: Authorizing `identity:get_user(user_id=338271833df8486b9ef50964b5ec7797)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:45.294882 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3995d4fe-64c5-4d7a-aec6-a237ab18bc57 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:45.295656 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 21/41] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:11:45 2022] GET /identity/v3/users/338271833df8486b9ef50964b5ec7797 => generated 247 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:11:45.301250 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995d4fe-64c5-4d7a-aec6-a237ab18bc57 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:45.315608 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:45.325964 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 21/42] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:11:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 15:11:45.349916 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a9707-7d69-42a0-8872-ad147a75697b None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:45.420798 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995d4fe-64c5-4d7a-aec6-a237ab18bc57 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:45.454131 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8371e453-57cf-4116-a339-539c08f6ca7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:45.454677 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8371e453-57cf-4116-a339-539c08f6ca7f None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:45.454889 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a9707-7d69-42a0-8872-ad147a75697b None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:45.454981 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8371e453-57cf-4116-a339-539c08f6ca7f None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:45.455224 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8371e453-57cf-4116-a339-539c08f6ca7f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:45.456410 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8371e453-57cf-4116-a339-539c08f6ca7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:45.457876 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8371e453-57cf-4116-a339-539c08f6ca7f None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:45.459450 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 22/43] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:11:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:11:45.465595 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8371e453-57cf-4116-a339-539c08f6ca7f None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:45.489894 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de537a-c5b9-4f9b-9b11-6491996346da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:45.490547 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de537a-c5b9-4f9b-9b11-6491996346da None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:45.490840 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de537a-c5b9-4f9b-9b11-6491996346da None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:45.491123 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de537a-c5b9-4f9b-9b11-6491996346da None admin] PATH_INFO: `/v3/domains/default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:45.492339 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de537a-c5b9-4f9b-9b11-6491996346da None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:45.493823 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de537a-c5b9-4f9b-9b11-6491996346da None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:45.495571 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 22/44] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:11:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:11:45.501596 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de537a-c5b9-4f9b-9b11-6491996346da None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:45.536561 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:45.543689 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:45.566226 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8371e453-57cf-4116-a339-539c08f6ca7f None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:45.605906 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276f9aa8-5ad6-4d01-9973-3e710b4dd505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:45.606628 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f9aa8-5ad6-4d01-9973-3e710b4dd505 None admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:45.606911 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f9aa8-5ad6-4d01-9973-3e710b4dd505 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:45.607193 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f9aa8-5ad6-4d01-9973-3e710b4dd505 None admin] PATH_INFO: `/v3/domains/default/users/338271833df8486b9ef50964b5ec7797/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:45.607741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de537a-c5b9-4f9b-9b11-6491996346da None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:45.609261 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276f9aa8-5ad6-4d01-9973-3e710b4dd505 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=338271833df8486b9ef50964b5ec7797, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:45.610742 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276f9aa8-5ad6-4d01-9973-3e710b4dd505 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:45.641330 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dda3d5-a43c-4543-adb6-3e95bc2edcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:45.641954 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dda3d5-a43c-4543-adb6-3e95bc2edcea None admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:45.642266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dda3d5-a43c-4543-adb6-3e95bc2edcea None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:45.642586 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dda3d5-a43c-4543-adb6-3e95bc2edcea None admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:45.643272 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dda3d5-a43c-4543-adb6-3e95bc2edcea None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:45.644754 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dda3d5-a43c-4543-adb6-3e95bc2edcea None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:45.722936 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 23/45] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 5 15:11:45 2022] POST /identity/v3/projects => generated 312 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:11:45.764771 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 23/46] 10.222.0.40 () {58 vars in 1361 bytes} [Fri Aug 5 15:11:45 2022] PUT /identity/v3/domains/default/users/338271833df8486b9ef50964b5ec7797/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:11:47.735289 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e4a25-a6b8-402f-8d5c-b3e38c087e84 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:47.735891 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e4a25-a6b8-402f-8d5c-b3e38c087e84 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:47.735891 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e4a25-a6b8-402f-8d5c-b3e38c087e84 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:47.737299 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 24/47] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:11:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:11:47.738257 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c731f-eb62-40c9-b1d9-c459303412cb None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:47.738642 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c731f-eb62-40c9-b1d9-c459303412cb None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:47.738833 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c731f-eb62-40c9-b1d9-c459303412cb None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:47.742009 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be039882-3af4-4146-94f0-801b30b614d2 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:47.742300 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be039882-3af4-4146-94f0-801b30b614d2 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:47.742585 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be039882-3af4-4146-94f0-801b30b614d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:47.743182 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 24/48] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:11:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 15:11:47.747974 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d539f1a-e5ab-4d0c-b49b-7775060971ab None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:47.748294 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d539f1a-e5ab-4d0c-b49b-7775060971ab None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:47.748579 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d539f1a-e5ab-4d0c-b49b-7775060971ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:47.760871 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-be039882-3af4-4146-94f0-801b30b614d2 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:11:47.762085 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-be039882-3af4-4146-94f0-801b30b614d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:11:47.767362 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-6d539f1a-e5ab-4d0c-b49b-7775060971ab None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:11:47.768894 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-6d539f1a-e5ab-4d0c-b49b-7775060971ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:11:47.828581 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 25/49] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:11:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 15:11:47.834595 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 25/50] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:11:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 15:11:47.851830 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be039882-3af4-4146-94f0-801b30b614d2 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:47.857867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d539f1a-e5ab-4d0c-b49b-7775060971ab None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:47.869247 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:47.921960 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:47.951377 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be039882-3af4-4146-94f0-801b30b614d2 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:47.956140 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d539f1a-e5ab-4d0c-b49b-7775060971ab None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:47.984548 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7b594-4eed-49c4-ab4e-2e9752286d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:47.985190 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7b594-4eed-49c4-ab4e-2e9752286d92 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:47.985540 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7b594-4eed-49c4-ab4e-2e9752286d92 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:47.985759 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7b594-4eed-49c4-ab4e-2e9752286d92 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:47.990135 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129c0e84-93cc-435f-90cc-2eca6c06404f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:47.990769 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c0e84-93cc-435f-90cc-2eca6c06404f None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:47.991052 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c0e84-93cc-435f-90cc-2eca6c06404f None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:47.991327 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c0e84-93cc-435f-90cc-2eca6c06404f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:47.992515 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129c0e84-93cc-435f-90cc-2eca6c06404f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:47.994098 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129c0e84-93cc-435f-90cc-2eca6c06404f None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:47.995672 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 26/51] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:11:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:11:47.995856 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7b594-4eed-49c4-ab4e-2e9752286d92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:11:48.001339 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-3ef7b594-4eed-49c4-ab4e-2e9752286d92 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:11:48.001339 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:11:48.001339 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:11:48.001339 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:11:48.001339 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:11:48.001339 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:11:48.001339 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:11:48.001339 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:11:48.001339 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:11:48.001339 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:11:48.001339 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:11:48.001339 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:11:48.001339 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:11:48.001339 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:11:48.001339 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:11:48.001339 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:11:48.002165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:11:48.002165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:11:48.002165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:11:48.002165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:11:48.002165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:11:48.002165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:11:48.002165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:11:48.002165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:11:48.002165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:11:48.002165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:11:48.002165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:11:48.002165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:11:48.002165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:11:48.002165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:11:48.002165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:11:48.002165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:11:48.002914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:11:48.002914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:11:48.002914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:11:48.002914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:11:48.002914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:11:48.002914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:11:48.002914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:11:48.002914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:11:48.002914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:11:48.002914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:11:48.002914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:11:48.002914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:11:48.003654 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129c0e84-93cc-435f-90cc-2eca6c06404f None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:48.003735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 26/52] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 15:11:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:11:48.008706 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7b594-4eed-49c4-ab4e-2e9752286d92 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:48.083103 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:48.097889 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129c0e84-93cc-435f-90cc-2eca6c06404f None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:48.105857 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7b594-4eed-49c4-ab4e-2e9752286d92 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:48.130148 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca93845d-2989-45f1-b768-77841b70e0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:48.130759 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93845d-2989-45f1-b768-77841b70e0fb None admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:48.131044 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93845d-2989-45f1-b768-77841b70e0fb None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:48.131323 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93845d-2989-45f1-b768-77841b70e0fb None admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:48.131994 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca93845d-2989-45f1-b768-77841b70e0fb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:48.134006 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca93845d-2989-45f1-b768-77841b70e0fb None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:48.139112 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:48.141439 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d0ca1a-b337-4b39-82e9-e6a2d0e91178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:48.142056 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0ca1a-b337-4b39-82e9-e6a2d0e91178 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:48.142370 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0ca1a-b337-4b39-82e9-e6a2d0e91178 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:48.142651 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0ca1a-b337-4b39-82e9-e6a2d0e91178 None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:48.143379 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d0ca1a-b337-4b39-82e9-e6a2d0e91178 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:11:48.143691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d0ca1a-b337-4b39-82e9-e6a2d0e91178 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:48.145144 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d0ca1a-b337-4b39-82e9-e6a2d0e91178 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:48.154067 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 27/53] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:11:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:11:48.159694 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d0ca1a-b337-4b39-82e9-e6a2d0e91178 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:48.220593 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 27/54] 10.222.0.40 () {60 vars in 1140 bytes} [Fri Aug 5 15:11:48 2022] POST /identity/v3/projects => generated 298 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:11:48.263857 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d0ca1a-b337-4b39-82e9-e6a2d0e91178 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:48.300762 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e269759-ce1e-4017-baf2-a80cfe0669ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:48.301433 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e269759-ce1e-4017-baf2-a80cfe0669ce None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:48.301729 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e269759-ce1e-4017-baf2-a80cfe0669ce None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:48.302011 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e269759-ce1e-4017-baf2-a80cfe0669ce None admin] PATH_INFO: `/v3/users/338271833df8486b9ef50964b5ec7797` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:48.303257 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e269759-ce1e-4017-baf2-a80cfe0669ce None admin] RBAC: Authorizing `identity:get_user(user_id=338271833df8486b9ef50964b5ec7797)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:48.304767 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e269759-ce1e-4017-baf2-a80cfe0669ce None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:48.306960 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 28/55] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:11:48 2022] GET /identity/v3/users/338271833df8486b9ef50964b5ec7797 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:11:48.312693 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca93845d-2989-45f1-b768-77841b70e0fb None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:48.417391 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca93845d-2989-45f1-b768-77841b70e0fb None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:48.451275 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1978a658-cdc4-43b1-9c80-af34d78946f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:48.451922 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978a658-cdc4-43b1-9c80-af34d78946f9 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:48.452244 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978a658-cdc4-43b1-9c80-af34d78946f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:48.452533 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978a658-cdc4-43b1-9c80-af34d78946f9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:48.453696 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1978a658-cdc4-43b1-9c80-af34d78946f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:48.455170 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1978a658-cdc4-43b1-9c80-af34d78946f9 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:48.461415 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 28/56] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:11:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:11:48.462613 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e269759-ce1e-4017-baf2-a80cfe0669ce None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:48.471663 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:48.564178 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e269759-ce1e-4017-baf2-a80cfe0669ce None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:48.598266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd0051-e76d-4a82-b2ef-11f9e42e763d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:48.598889 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd0051-e76d-4a82-b2ef-11f9e42e763d None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:48.599180 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd0051-e76d-4a82-b2ef-11f9e42e763d None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:48.599459 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd0051-e76d-4a82-b2ef-11f9e42e763d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:48.600313 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbd0051-e76d-4a82-b2ef-11f9e42e763d None admin] RBAC: Adding query filter params (role.id=7c88c815db854954b516ab97cd471647, scope.domain.id=default, user.id=338271833df8486b9ef50964b5ec7797) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:11:48.600664 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbd0051-e76d-4a82-b2ef-11f9e42e763d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:48.602129 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbd0051-e76d-4a82-b2ef-11f9e42e763d None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:48.607991 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 29/57] 10.222.0.40 () {56 vars in 1322 bytes} [Fri Aug 5 15:11:48 2022] GET /identity/v3/role_assignments?user.id=338271833df8486b9ef50964b5ec7797&scope.domain.id=default&role.id=7c88c815db854954b516ab97cd471647 => generated 530 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:11:50.226727 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318c3ac-cbee-4696-b0e8-2fae6cb61581 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:50.227715 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318c3ac-cbee-4696-b0e8-2fae6cb61581 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:50.227798 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318c3ac-cbee-4696-b0e8-2fae6cb61581 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:50.229179 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 29/58] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:11:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:11:50.234258 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c11718-8ba3-4f6c-83a6-97fc2096cde9 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:50.234657 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c11718-8ba3-4f6c-83a6-97fc2096cde9 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:50.234817 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c11718-8ba3-4f6c-83a6-97fc2096cde9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:50.253309 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-27c11718-8ba3-4f6c-83a6-97fc2096cde9 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:11:50.254475 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-27c11718-8ba3-4f6c-83a6-97fc2096cde9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:11:50.313674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:50.317933 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 30/59] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:11:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:11:50.341625 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e318c3ac-cbee-4696-b0e8-2fae6cb61581 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:50.396966 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:50.441316 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e318c3ac-cbee-4696-b0e8-2fae6cb61581 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:50.473463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ef27b5-fb3b-45cd-9491-d69eababf665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:50.474089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ef27b5-fb3b-45cd-9491-d69eababf665 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:50.474378 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ef27b5-fb3b-45cd-9491-d69eababf665 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:50.474662 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ef27b5-fb3b-45cd-9491-d69eababf665 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:50.475813 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ef27b5-fb3b-45cd-9491-d69eababf665 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:50.477844 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ef27b5-fb3b-45cd-9491-d69eababf665 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:50.479407 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 30/60] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:11:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:11:50.484957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c11718-8ba3-4f6c-83a6-97fc2096cde9 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:50.581129 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c11718-8ba3-4f6c-83a6-97fc2096cde9 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:50.614515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42952e05-468b-4700-98e9-fe73d6b417dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:50.615123 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42952e05-468b-4700-98e9-fe73d6b417dc None admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:50.615411 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42952e05-468b-4700-98e9-fe73d6b417dc None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:50.615692 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42952e05-468b-4700-98e9-fe73d6b417dc None admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:50.616425 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42952e05-468b-4700-98e9-fe73d6b417dc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:50.617940 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42952e05-468b-4700-98e9-fe73d6b417dc None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:50.698143 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 31/61] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 5 15:11:50 2022] POST /identity/v3/users => generated 275 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:11:53.745308 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df485b32-4525-4dfb-9582-81e4feb9963d None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:53.746502 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df485b32-4525-4dfb-9582-81e4feb9963d None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:53.747000 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df485b32-4525-4dfb-9582-81e4feb9963d None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:53.748605 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 31/62] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:11:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:11:53.754403 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a37ba7-90d9-49dc-a760-960cc3dea421 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:53.755204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a37ba7-90d9-49dc-a760-960cc3dea421 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:53.755692 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a37ba7-90d9-49dc-a760-960cc3dea421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:53.780327 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-75a37ba7-90d9-49dc-a760-960cc3dea421 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:11:53.781536 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-75a37ba7-90d9-49dc-a760-960cc3dea421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:11:53.853149 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:53.873633 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 32/63] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:11:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:11:53.899157 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc025a-539b-4052-891d-b56e137afe59 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:53.899764 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc025a-539b-4052-891d-b56e137afe59 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:53.900299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc025a-539b-4052-891d-b56e137afe59 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:53.901784 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a37ba7-90d9-49dc-a760-960cc3dea421 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:53.901909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 32/64] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:11:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:11:53.907486 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6ebe6-a556-4cde-b839-381b62898e00 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:53.908030 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6ebe6-a556-4cde-b839-381b62898e00 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:53.908400 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6ebe6-a556-4cde-b839-381b62898e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:53.937874 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-f7a6ebe6-a556-4cde-b839-381b62898e00 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:11:53.939102 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-f7a6ebe6-a556-4cde-b839-381b62898e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:11:54.076439 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a37ba7-90d9-49dc-a760-960cc3dea421 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:54.096164 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 33/65] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:11:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 15:11:54.097776 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11335c80-3d53-447f-8fe7-93317b31eea7 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:54.098202 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11335c80-3d53-447f-8fe7-93317b31eea7 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:54.098618 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11335c80-3d53-447f-8fe7-93317b31eea7 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:54.100328 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 34/66] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:11:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:11:54.105082 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed02fce-7230-429e-801c-1c9353ed470b None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:54.105374 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed02fce-7230-429e-801c-1c9353ed470b None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:54.105647 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed02fce-7230-429e-801c-1c9353ed470b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:54.123337 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4321c9-8a81-4854-b515-809ed3bbc5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:54.124094 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4321c9-8a81-4854-b515-809ed3bbc5c2 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:54.124386 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4321c9-8a81-4854-b515-809ed3bbc5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:54.124670 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4321c9-8a81-4854-b515-809ed3bbc5c2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:54.132210 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-3ed02fce-7230-429e-801c-1c9353ed470b None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:11:54.134217 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-3ed02fce-7230-429e-801c-1c9353ed470b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:11:54.140564 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4321c9-8a81-4854-b515-809ed3bbc5c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:11:54.145602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-1f4321c9-8a81-4854-b515-809ed3bbc5c2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 15:11:54.145602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:11:54.145602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:11:54.145602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:11:54.145602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:11:54.145602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:11:54.145602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:11:54.145602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:11:54.145602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:11:54.145602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:11:54.145602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:11:54.145602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:11:54.145602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:11:54.145602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:11:54.145602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:11:54.145602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:11:54.146456 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:11:54.146456 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:11:54.146456 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:11:54.146456 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:11:54.146456 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:11:54.146456 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:11:54.146456 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:11:54.146456 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:11:54.146456 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:11:54.146456 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:11:54.146456 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:11:54.146456 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:11:54.146456 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:11:54.146456 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:11:54.146456 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:11:54.146456 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:11:54.147215 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:11:54.147215 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:11:54.147215 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:11:54.147215 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:11:54.147215 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:11:54.147215 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:11:54.147215 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:11:54.147215 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:11:54.147215 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:11:54.147215 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:11:54.147215 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 15:11:54.147215 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:11:54.147215 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 33/67] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:11:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:11:54.147835 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4321c9-8a81-4854-b515-809ed3bbc5c2 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:54.196426 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:54.218490 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:54.221773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 35/68] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:11:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:11:54.228228 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed02fce-7230-429e-801c-1c9353ed470b None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:54.302154 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4321c9-8a81-4854-b515-809ed3bbc5c2 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:54.341103 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de31aed-d6c0-48eb-9717-591eec20725e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:54.342397 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de31aed-d6c0-48eb-9717-591eec20725e None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:54.343114 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de31aed-d6c0-48eb-9717-591eec20725e None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:54.343681 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de31aed-d6c0-48eb-9717-591eec20725e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:54.351052 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed02fce-7230-429e-801c-1c9353ed470b None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:54.358078 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de31aed-d6c0-48eb-9717-591eec20725e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:11:54.365589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-2de31aed-d6c0-48eb-9717-591eec20725e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:11:54.365589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:11:54.365589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:11:54.365589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:11:54.365589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:11:54.365589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:11:54.365589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:11:54.365589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:11:54.365589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:11:54.365589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:11:54.365589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:11:54.365589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:11:54.365589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:11:54.365589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:11:54.365589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:11:54.365589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:11:54.366461 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:11:54.366461 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:11:54.366461 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:11:54.366461 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:11:54.366461 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:11:54.366461 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:11:54.366461 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:11:54.366461 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:11:54.366461 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:11:54.366461 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:11:54.366461 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:11:54.366461 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:11:54.366461 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:11:54.366461 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:11:54.366461 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:11:54.366461 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:11:54.367830 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:11:54.367830 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:11:54.367830 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:11:54.367830 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:11:54.367830 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:11:54.367830 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:11:54.367830 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:11:54.367830 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:11:54.367830 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:11:54.367830 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:11:54.367830 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:11:54.367830 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:11:54.367830 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 34/69] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 15:11:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:11:54.368587 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de31aed-d6c0-48eb-9717-591eec20725e None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:54.391696 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e703dd40-a217-4c39-8c05-2deedc71ec3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:54.392352 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703dd40-a217-4c39-8c05-2deedc71ec3e None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:54.392637 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703dd40-a217-4c39-8c05-2deedc71ec3e None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:54.392920 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703dd40-a217-4c39-8c05-2deedc71ec3e None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:54.393631 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e703dd40-a217-4c39-8c05-2deedc71ec3e None admin] RBAC: Adding query filter params (name=member) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:11:54.393942 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e703dd40-a217-4c39-8c05-2deedc71ec3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:54.395311 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e703dd40-a217-4c39-8c05-2deedc71ec3e None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:54.404563 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 36/70] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 15:11:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:11:54.405803 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b66d0-f5a2-4437-ab3b-e3eaabc353c9 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:54.406095 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b66d0-f5a2-4437-ab3b-e3eaabc353c9 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:54.406373 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b66d0-f5a2-4437-ab3b-e3eaabc353c9 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:54.411608 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 37/71] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:11:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:11:54.411608 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584b66d0-f5a2-4437-ab3b-e3eaabc353c9 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:54.458068 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:54.488257 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de31aed-d6c0-48eb-9717-591eec20725e None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:54.495861 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:54.513528 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584b66d0-f5a2-4437-ab3b-e3eaabc353c9 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:54.523253 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad52d6-978e-47c4-abde-38248dca8d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:54.523938 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad52d6-978e-47c4-abde-38248dca8d42 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:54.524255 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad52d6-978e-47c4-abde-38248dca8d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:54.524532 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad52d6-978e-47c4-abde-38248dca8d42 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:54.535446 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aad52d6-978e-47c4-abde-38248dca8d42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:11:54.540735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-3aad52d6-978e-47c4-abde-38248dca8d42 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 15:11:54.540735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:11:54.540735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:11:54.540735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:11:54.540735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:11:54.540735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:11:54.540735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:11:54.540735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:11:54.540735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:11:54.540735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:11:54.540735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:11:54.540735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:11:54.540735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:11:54.540735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:11:54.540735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:11:54.540735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:11:54.541647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:11:54.541647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:11:54.541647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:11:54.541647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:11:54.541647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:11:54.541647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:11:54.541647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:11:54.541647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:11:54.541647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:11:54.541647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:11:54.541647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:11:54.541647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:11:54.541647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:11:54.541647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:11:54.541647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:11:54.541647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:11:54.542437 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:11:54.542437 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:11:54.542437 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:11:54.542437 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:11:54.542437 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:11:54.542437 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:11:54.542437 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:11:54.542437 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:11:54.542437 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:11:54.542437 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:11:54.542437 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 15:11:54.542437 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:11:54.542437 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 35/72] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 15:11:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:11:54.543065 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad52d6-978e-47c4-abde-38248dca8d42 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:54.548747 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19e4869-79ea-4083-97e8-67501fbcb4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:54.549364 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e4869-79ea-4083-97e8-67501fbcb4a3 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:54.549650 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e4869-79ea-4083-97e8-67501fbcb4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:54.549925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e4869-79ea-4083-97e8-67501fbcb4a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:54.550638 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19e4869-79ea-4083-97e8-67501fbcb4a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:11:54.550992 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19e4869-79ea-4083-97e8-67501fbcb4a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:54.552329 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19e4869-79ea-4083-97e8-67501fbcb4a3 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:54.561518 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 38/73] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:11:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:11:54.567125 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a29059-86dc-4d70-8fec-6e057b147c1f None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:54.567414 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a29059-86dc-4d70-8fec-6e057b147c1f None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:54.567705 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a29059-86dc-4d70-8fec-6e057b147c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:54.584986 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-68a29059-86dc-4d70-8fec-6e057b147c1f None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:11:54.586187 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-68a29059-86dc-4d70-8fec-6e057b147c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:11:54.640802 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad52d6-978e-47c4-abde-38248dca8d42 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:54.653550 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 39/74] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:11:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:11:54.654866 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c77f02-3b69-47ac-8f28-f53735756938 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:54.655162 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c77f02-3b69-47ac-8f28-f53735756938 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:54.655442 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c77f02-3b69-47ac-8f28-f53735756938 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:54.656741 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 40/75] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:11:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:11:54.657450 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c77f02-3b69-47ac-8f28-f53735756938 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:54.685343 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ac061-efa7-4173-b99c-8cec4a437649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:54.685989 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ac061-efa7-4173-b99c-8cec4a437649 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:54.686274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ac061-efa7-4173-b99c-8cec4a437649 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:54.686552 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ac061-efa7-4173-b99c-8cec4a437649 None admin] PATH_INFO: `/v3/users/a6cf4078f01748e98571f285e560294e` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:54.699686 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284ac061-efa7-4173-b99c-8cec4a437649 None admin] RBAC: Authorizing `identity:get_user(user_id=a6cf4078f01748e98571f285e560294e)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:54.700465 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:54.701235 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284ac061-efa7-4173-b99c-8cec4a437649 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:54.702868 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 36/76] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:11:54 2022] GET /identity/v3/users/a6cf4078f01748e98571f285e560294e => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:11:54.703629 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ac061-efa7-4173-b99c-8cec4a437649 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:54.727739 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:54.765534 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c77f02-3b69-47ac-8f28-f53735756938 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:54.798816 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3325ff6b-2642-49dc-8dc7-baf14e1b8aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:54.799437 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325ff6b-2642-49dc-8dc7-baf14e1b8aba None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:54.799727 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325ff6b-2642-49dc-8dc7-baf14e1b8aba None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:54.800008 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325ff6b-2642-49dc-8dc7-baf14e1b8aba None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:54.800758 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3325ff6b-2642-49dc-8dc7-baf14e1b8aba None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:11:54.801077 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3325ff6b-2642-49dc-8dc7-baf14e1b8aba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:54.801201 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ac061-efa7-4173-b99c-8cec4a437649 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:54.802454 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3325ff6b-2642-49dc-8dc7-baf14e1b8aba None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:54.810988 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 41/77] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:11:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:11:54.811938 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174e935-710d-4b78-913a-438c67577971 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:54.812269 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174e935-710d-4b78-913a-438c67577971 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:54.812555 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174e935-710d-4b78-913a-438c67577971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:54.830059 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-6174e935-710d-4b78-913a-438c67577971 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:11:54.831287 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-6174e935-710d-4b78-913a-438c67577971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:11:54.836847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c92ef3-6d96-470d-8594-1f7e7279ba5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:54.837486 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c92ef3-6d96-470d-8594-1f7e7279ba5a None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:54.837771 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c92ef3-6d96-470d-8594-1f7e7279ba5a None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:54.838049 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c92ef3-6d96-470d-8594-1f7e7279ba5a None admin] PATH_INFO: `/v3/users/338271833df8486b9ef50964b5ec7797` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:54.839229 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c92ef3-6d96-470d-8594-1f7e7279ba5a None admin] RBAC: Authorizing `identity:get_user(user_id=338271833df8486b9ef50964b5ec7797)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:54.840749 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c92ef3-6d96-470d-8594-1f7e7279ba5a None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:54.842351 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 37/78] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:11:54 2022] GET /identity/v3/users/338271833df8486b9ef50964b5ec7797 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:11:54.845049 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c92ef3-6d96-470d-8594-1f7e7279ba5a None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:54.897631 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 42/79] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:11:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:11:54.898413 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6174e935-710d-4b78-913a-438c67577971 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:54.925892 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:54.951480 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:54.959982 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c92ef3-6d96-470d-8594-1f7e7279ba5a None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:54.994430 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08410b38-d63f-40ae-b551-ff8ac8310d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:54.995070 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08410b38-d63f-40ae-b551-ff8ac8310d6c None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:54.995352 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08410b38-d63f-40ae-b551-ff8ac8310d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:54.995630 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08410b38-d63f-40ae-b551-ff8ac8310d6c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:54.997270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6174e935-710d-4b78-913a-438c67577971 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:55.005884 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08410b38-d63f-40ae-b551-ff8ac8310d6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:11:55.010742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-08410b38-d63f-40ae-b551-ff8ac8310d6c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 15:11:55.010742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:11:55.010742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:11:55.010742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:11:55.010742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:11:55.010742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:11:55.010742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:11:55.010742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:11:55.010742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:11:55.010742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:11:55.010742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:11:55.010742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:11:55.010742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:11:55.010742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:11:55.010742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:11:55.010742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:11:55.011662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:11:55.011662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:11:55.011662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:11:55.011662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:11:55.011662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:11:55.011662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:11:55.011662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:11:55.011662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:11:55.011662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:11:55.011662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:11:55.011662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:11:55.011662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:11:55.011662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:11:55.011662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:11:55.011662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:11:55.011662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:11:55.012476 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:11:55.012476 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:11:55.012476 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:11:55.012476 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:11:55.012476 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:11:55.012476 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:11:55.012476 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:11:55.012476 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:11:55.012476 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:11:55.012476 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:11:55.012476 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 15:11:55.012476 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:11:55.012476 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 38/80] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:11:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:11:55.013123 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08410b38-d63f-40ae-b551-ff8ac8310d6c None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:55.035143 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38799fbf-f64a-4bfe-8c57-6b968555d252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:55.035760 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38799fbf-f64a-4bfe-8c57-6b968555d252 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:55.036048 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38799fbf-f64a-4bfe-8c57-6b968555d252 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:55.036352 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38799fbf-f64a-4bfe-8c57-6b968555d252 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:55.102372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38799fbf-f64a-4bfe-8c57-6b968555d252 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:55.103857 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38799fbf-f64a-4bfe-8c57-6b968555d252 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:55.112466 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08410b38-d63f-40ae-b551-ff8ac8310d6c None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:55.130396 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:55.145292 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c33c3ff-eb7c-4258-8142-778256303e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:55.145939 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33c3ff-eb7c-4258-8142-778256303e27 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:55.146235 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33c3ff-eb7c-4258-8142-778256303e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:55.146523 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33c3ff-eb7c-4258-8142-778256303e27 None admin] PATH_INFO: `/v3/users/a6cf4078f01748e98571f285e560294e` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:55.147784 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c33c3ff-eb7c-4258-8142-778256303e27 None admin] RBAC: Authorizing `identity:get_user(user_id=a6cf4078f01748e98571f285e560294e)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:55.149758 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c33c3ff-eb7c-4258-8142-778256303e27 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:55.151358 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 39/81] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:11:55 2022] GET /identity/v3/users/a6cf4078f01748e98571f285e560294e => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:11:55.156154 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c33c3ff-eb7c-4258-8142-778256303e27 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:55.179186 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:55.202560 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 43/82] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:11:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 15:11:55.203392 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38799fbf-f64a-4bfe-8c57-6b968555d252 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:55.261137 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c33c3ff-eb7c-4258-8142-778256303e27 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:55.294700 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af418e2-3366-48e2-9385-9de290d1c3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:55.295319 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af418e2-3366-48e2-9385-9de290d1c3e0 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:55.295604 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af418e2-3366-48e2-9385-9de290d1c3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:55.295885 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af418e2-3366-48e2-9385-9de290d1c3e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:55.301711 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38799fbf-f64a-4bfe-8c57-6b968555d252 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:55.326982 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:55.334362 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb2c09c-24ed-4e1a-a484-7a73910fcda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:55.335005 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2c09c-24ed-4e1a-a484-7a73910fcda5 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:55.335301 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2c09c-24ed-4e1a-a484-7a73910fcda5 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:55.352702 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2c09c-24ed-4e1a-a484-7a73910fcda5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:55.352702 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb2c09c-24ed-4e1a-a484-7a73910fcda5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:55.352702 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb2c09c-24ed-4e1a-a484-7a73910fcda5 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:55.352702 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 44/83] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:11:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:11:55.352702 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb2c09c-24ed-4e1a-a484-7a73910fcda5 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:55.362917 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af418e2-3366-48e2-9385-9de290d1c3e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:55.364467 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af418e2-3366-48e2-9385-9de290d1c3e0 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:55.384522 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:55.443550 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb2c09c-24ed-4e1a-a484-7a73910fcda5 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:55.463026 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 40/84] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:11:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 15:11:55.464158 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af418e2-3366-48e2-9385-9de290d1c3e0 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:55.481530 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a1937-5496-4994-8115-58ec4dd61774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:55.481938 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a1937-5496-4994-8115-58ec4dd61774 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:55.482225 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a1937-5496-4994-8115-58ec4dd61774 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:55.482536 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a1937-5496-4994-8115-58ec4dd61774 None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:55.483246 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41a1937-5496-4994-8115-58ec4dd61774 None admin] RBAC: Adding query filter params (name=member) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:11:55.483560 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41a1937-5496-4994-8115-58ec4dd61774 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:55.484968 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41a1937-5496-4994-8115-58ec4dd61774 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:55.497253 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 45/85] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 15:11:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:11:55.497253 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a1937-5496-4994-8115-58ec4dd61774 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:55.549523 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:55.565364 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af418e2-3366-48e2-9385-9de290d1c3e0 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:55.586487 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:55.591186 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a1937-5496-4994-8115-58ec4dd61774 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:55.600802 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb4103-9100-44e6-92c7-47dc33dc1e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:55.601452 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb4103-9100-44e6-92c7-47dc33dc1e39 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:55.601752 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb4103-9100-44e6-92c7-47dc33dc1e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:55.602031 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb4103-9100-44e6-92c7-47dc33dc1e39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:55.626064 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f15b78b-cd63-43f3-b652-2399296c50e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:55.626714 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15b78b-cd63-43f3-b652-2399296c50e5 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:55.626997 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15b78b-cd63-43f3-b652-2399296c50e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:55.627276 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15b78b-cd63-43f3-b652-2399296c50e5 None admin] PATH_INFO: `/v3/projects/b972ca1ff8ee4c558eeb45ec24303061` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:55.628686 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f15b78b-cd63-43f3-b652-2399296c50e5 None admin] RBAC: Authorizing `identity:get_project(project_id=b972ca1ff8ee4c558eeb45ec24303061)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:55.630408 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f15b78b-cd63-43f3-b652-2399296c50e5 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:55.632223 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 46/86] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:11:55 2022] GET /identity/v3/projects/b972ca1ff8ee4c558eeb45ec24303061 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:11:55.633541 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f15b78b-cd63-43f3-b652-2399296c50e5 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:55.674750 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cb4103-9100-44e6-92c7-47dc33dc1e39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:55.676293 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cb4103-9100-44e6-92c7-47dc33dc1e39 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:55.735786 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f15b78b-cd63-43f3-b652-2399296c50e5 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:55.761485 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:55.769866 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9366e0-75f7-4bc2-80de-ea98f2cd29ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:55.770484 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9366e0-75f7-4bc2-80de-ea98f2cd29ca None admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:55.774097 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9366e0-75f7-4bc2-80de-ea98f2cd29ca None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:55.774097 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9366e0-75f7-4bc2-80de-ea98f2cd29ca None admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:55.774319 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 41/87] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:11:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 15:11:55.774420 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9366e0-75f7-4bc2-80de-ea98f2cd29ca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:55.774491 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb4103-9100-44e6-92c7-47dc33dc1e39 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:55.774567 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9366e0-75f7-4bc2-80de-ea98f2cd29ca None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:55.795430 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:55.852229 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 47/88] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 5 15:11:55 2022] POST /identity/v3/users => generated 289 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:11:55.853455 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9366e0-75f7-4bc2-80de-ea98f2cd29ca None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:55.877048 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb4103-9100-44e6-92c7-47dc33dc1e39 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:55.915497 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877f299-4252-4312-952c-789be8fc598d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:55.916203 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877f299-4252-4312-952c-789be8fc598d None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:55.916491 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877f299-4252-4312-952c-789be8fc598d None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:55.916771 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877f299-4252-4312-952c-789be8fc598d None admin] PATH_INFO: `/v3/projects/b972ca1ff8ee4c558eeb45ec24303061` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:55.918005 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d877f299-4252-4312-952c-789be8fc598d None admin] RBAC: Authorizing `identity:get_project(project_id=b972ca1ff8ee4c558eeb45ec24303061)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:55.919505 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d877f299-4252-4312-952c-789be8fc598d None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:55.926403 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 42/89] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:11:55 2022] GET /identity/v3/projects/b972ca1ff8ee4c558eeb45ec24303061 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:11:55.926403 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877f299-4252-4312-952c-789be8fc598d None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:55.987440 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9366e0-75f7-4bc2-80de-ea98f2cd29ca None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:56.023678 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:56.029863 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541cd2c-a4be-4c72-bd15-29220ddc9009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:56.029863 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541cd2c-a4be-4c72-bd15-29220ddc9009 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:56.029863 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541cd2c-a4be-4c72-bd15-29220ddc9009 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:56.029863 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541cd2c-a4be-4c72-bd15-29220ddc9009 None admin] PATH_INFO: `/v3/users/a6cf4078f01748e98571f285e560294e` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:56.030458 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3541cd2c-a4be-4c72-bd15-29220ddc9009 None admin] RBAC: Authorizing `identity:get_user(user_id=a6cf4078f01748e98571f285e560294e)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:56.031908 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3541cd2c-a4be-4c72-bd15-29220ddc9009 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:56.033761 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 48/90] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:11:55 2022] GET /identity/v3/users/a6cf4078f01748e98571f285e560294e => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:11:56.034538 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541cd2c-a4be-4c72-bd15-29220ddc9009 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:56.039041 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877f299-4252-4312-952c-789be8fc598d None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:56.079243 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01c3d0-526c-4c94-bb84-cbd9903df232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:56.080290 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01c3d0-526c-4c94-bb84-cbd9903df232 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:56.081895 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01c3d0-526c-4c94-bb84-cbd9903df232 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:56.082189 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01c3d0-526c-4c94-bb84-cbd9903df232 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:56.083003 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01c3d0-526c-4c94-bb84-cbd9903df232 None admin] RBAC: Adding query filter params (role.id=d6c91838e4944cfbb23e1b6ecb009d26, scope.project.id=b972ca1ff8ee4c558eeb45ec24303061, user.id=a6cf4078f01748e98571f285e560294e) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:11:56.083325 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01c3d0-526c-4c94-bb84-cbd9903df232 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:56.085754 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01c3d0-526c-4c94-bb84-cbd9903df232 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:56.092370 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 43/91] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:11:55 2022] GET /identity/v3/role_assignments?user.id=a6cf4078f01748e98571f285e560294e&scope.project.id=b972ca1ff8ee4c558eeb45ec24303061&role.id=d6c91838e4944cfbb23e1b6ecb009d26 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:11:56.093124 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01c3d0-526c-4c94-bb84-cbd9903df232 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:56.102336 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:56.182709 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541cd2c-a4be-4c72-bd15-29220ddc9009 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:56.243468 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e45fc-c15c-40cb-b95e-7c95e7465b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:56.244254 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e45fc-c15c-40cb-b95e-7c95e7465b2a None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:56.244647 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e45fc-c15c-40cb-b95e-7c95e7465b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:56.245053 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e45fc-c15c-40cb-b95e-7c95e7465b2a None admin] PATH_INFO: `/v3/projects/b972ca1ff8ee4c558eeb45ec24303061` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:56.245991 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01c3d0-526c-4c94-bb84-cbd9903df232 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:56.246555 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e45fc-c15c-40cb-b95e-7c95e7465b2a None admin] RBAC: Authorizing `identity:get_project(project_id=b972ca1ff8ee4c558eeb45ec24303061)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:56.248555 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e45fc-c15c-40cb-b95e-7c95e7465b2a None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:56.251969 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 49/92] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:11:56 2022] GET /identity/v3/projects/b972ca1ff8ee4c558eeb45ec24303061 => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:11:56.252770 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e45fc-c15c-40cb-b95e-7c95e7465b2a None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:56.296950 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd1430-ee32-4a69-a3c0-d52feef2796a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:56.297862 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd1430-ee32-4a69-a3c0-d52feef2796a None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:56.298365 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd1430-ee32-4a69-a3c0-d52feef2796a None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:56.298908 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd1430-ee32-4a69-a3c0-d52feef2796a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:56.299930 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd1430-ee32-4a69-a3c0-d52feef2796a None admin] RBAC: Adding query filter params (role.id=7c88c815db854954b516ab97cd471647, scope.project.id=b972ca1ff8ee4c558eeb45ec24303061, user.id=338271833df8486b9ef50964b5ec7797) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:11:56.300499 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd1430-ee32-4a69-a3c0-d52feef2796a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:56.302216 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd1430-ee32-4a69-a3c0-d52feef2796a None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:56.309681 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 44/93] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:11:56 2022] GET /identity/v3/role_assignments?user.id=338271833df8486b9ef50964b5ec7797&scope.project.id=b972ca1ff8ee4c558eeb45ec24303061&role.id=7c88c815db854954b516ab97cd471647 => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:11:56.310734 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd1430-ee32-4a69-a3c0-d52feef2796a None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:56.353008 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:56.443031 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e45fc-c15c-40cb-b95e-7c95e7465b2a None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:56.489708 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:56.508708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd1430-ee32-4a69-a3c0-d52feef2796a None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:56.534395 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd89b10-bd6f-467d-b55f-4aa2332fa8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:56.535040 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd89b10-bd6f-467d-b55f-4aa2332fa8b8 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:56.535327 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd89b10-bd6f-467d-b55f-4aa2332fa8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:56.535607 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd89b10-bd6f-467d-b55f-4aa2332fa8b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:56.553432 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c21cc-35b6-4591-8c25-65fbeb998e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:56.554078 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c21cc-35b6-4591-8c25-65fbeb998e21 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:56.554365 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c21cc-35b6-4591-8c25-65fbeb998e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:56.555215 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c21cc-35b6-4591-8c25-65fbeb998e21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:56.556027 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212c21cc-35b6-4591-8c25-65fbeb998e21 None admin] RBAC: Adding query filter params (role.id=944427ca14ee4c81a05ec67e54f50e7f, scope.project.id=b972ca1ff8ee4c558eeb45ec24303061, user.id=a6cf4078f01748e98571f285e560294e) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:11:56.556467 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212c21cc-35b6-4591-8c25-65fbeb998e21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:56.557946 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212c21cc-35b6-4591-8c25-65fbeb998e21 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:56.569590 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 45/94] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:11:56 2022] GET /identity/v3/role_assignments?user.id=a6cf4078f01748e98571f285e560294e&scope.project.id=b972ca1ff8ee4c558eeb45ec24303061&role.id=944427ca14ee4c81a05ec67e54f50e7f => generated 259 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:11:56.668168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd89b10-bd6f-467d-b55f-4aa2332fa8b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:56.669748 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd89b10-bd6f-467d-b55f-4aa2332fa8b8 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:56.791464 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 50/95] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:11:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 15:11:56.797255 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c21cc-35b6-4591-8c25-65fbeb998e21 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:56.934396 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:56.955928 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c21cc-35b6-4591-8c25-65fbeb998e21 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:57.000518 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c479e5fb-1251-44d0-9d82-d26b66a63eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:57.001233 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479e5fb-1251-44d0-9d82-d26b66a63eac None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:57.001531 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479e5fb-1251-44d0-9d82-d26b66a63eac None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:57.001817 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479e5fb-1251-44d0-9d82-d26b66a63eac None admin] PATH_INFO: `/v3/projects/eab4ef6110f1406cbd92b046713d575b` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:57.003043 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c479e5fb-1251-44d0-9d82-d26b66a63eac None admin] RBAC: Authorizing `identity:get_project(project_id=eab4ef6110f1406cbd92b046713d575b)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:57.004547 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c479e5fb-1251-44d0-9d82-d26b66a63eac None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:57.006237 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 46/96] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:11:56 2022] GET /identity/v3/projects/eab4ef6110f1406cbd92b046713d575b => generated 312 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:11:57.012679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd89b10-bd6f-467d-b55f-4aa2332fa8b8 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:57.036856 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:57.145297 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd89b10-bd6f-467d-b55f-4aa2332fa8b8 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:57.195683 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15295bc5-de05-4ff9-969a-0f14c3c3c6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:57.196342 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15295bc5-de05-4ff9-969a-0f14c3c3c6ff None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:57.196636 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15295bc5-de05-4ff9-969a-0f14c3c3c6ff None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:57.196922 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15295bc5-de05-4ff9-969a-0f14c3c3c6ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:57.197726 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15295bc5-de05-4ff9-969a-0f14c3c3c6ff None admin] RBAC: Adding query filter params (role.id=d6c91838e4944cfbb23e1b6ecb009d26, scope.project.id=eab4ef6110f1406cbd92b046713d575b, user.id=a6cf4078f01748e98571f285e560294e) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:11:57.198045 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15295bc5-de05-4ff9-969a-0f14c3c3c6ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:57.199510 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15295bc5-de05-4ff9-969a-0f14c3c3c6ff None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:57.212201 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 51/97] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:11:57 2022] GET /identity/v3/role_assignments?user.id=a6cf4078f01748e98571f285e560294e&scope.project.id=eab4ef6110f1406cbd92b046713d575b&role.id=d6c91838e4944cfbb23e1b6ecb009d26 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:11:58.784933 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90be18-a109-4066-8627-55348f47325a None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:58.785448 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90be18-a109-4066-8627-55348f47325a None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:58.785524 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90be18-a109-4066-8627-55348f47325a None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:58.786860 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 47/98] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:11:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:11:58.792248 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a29da4-97a0-428c-a65e-60d1ddb12c10 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:58.792643 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a29da4-97a0-428c-a65e-60d1ddb12c10 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:58.792849 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a29da4-97a0-428c-a65e-60d1ddb12c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:58.811723 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-17a29da4-97a0-428c-a65e-60d1ddb12c10 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:11:58.812985 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-17a29da4-97a0-428c-a65e-60d1ddb12c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:11:58.816536 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032b550-9bc4-42f8-b70c-df2a054f86d0 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:58.816829 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032b550-9bc4-42f8-b70c-df2a054f86d0 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:58.817104 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032b550-9bc4-42f8-b70c-df2a054f86d0 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:58.818456 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 48/99] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:11:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:11:58.824146 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc9d98-c85e-4728-97ab-e46e6b4db60f None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:58.824438 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc9d98-c85e-4728-97ab-e46e6b4db60f None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:58.824725 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc9d98-c85e-4728-97ab-e46e6b4db60f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:58.852599 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-86bc9d98-c85e-4728-97ab-e46e6b4db60f None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:11:58.853815 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-86bc9d98-c85e-4728-97ab-e46e6b4db60f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:11:58.948172 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 52/100] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:11:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:11:58.971073 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a29da4-97a0-428c-a65e-60d1ddb12c10 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:58.983916 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 49/101] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:11:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 15:11:58.985956 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:59.001984 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524a5d4-40bf-4f0c-ac39-006bb391843c None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:59.002289 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524a5d4-40bf-4f0c-ac39-006bb391843c None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:59.002555 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524a5d4-40bf-4f0c-ac39-006bb391843c None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:59.004190 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 50/102] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:11:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 15:11:59.014967 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3648b2-fb6e-42e9-bf2d-6b9d99cf515e None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:59.015298 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3648b2-fb6e-42e9-bf2d-6b9d99cf515e None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:59.015529 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3648b2-fb6e-42e9-bf2d-6b9d99cf515e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:59.043743 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-7e3648b2-fb6e-42e9-bf2d-6b9d99cf515e None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:11:59.045612 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-7e3648b2-fb6e-42e9-bf2d-6b9d99cf515e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:11:59.091388 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a29da4-97a0-428c-a65e-60d1ddb12c10 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:59.155986 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6329851f-412b-4633-8c44-f9cae5b55d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:59.156664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6329851f-412b-4633-8c44-f9cae5b55d4e None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:59.156955 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6329851f-412b-4633-8c44-f9cae5b55d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:59.157239 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6329851f-412b-4633-8c44-f9cae5b55d4e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:59.159837 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 51/103] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:11:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 15:11:59.161781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3648b2-fb6e-42e9-bf2d-6b9d99cf515e None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:59.181730 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6329851f-412b-4633-8c44-f9cae5b55d4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:11:59.186739 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:59.191420 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-6329851f-412b-4633-8c44-f9cae5b55d4e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 15:11:59.191420 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:11:59.191420 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:11:59.191420 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:11:59.191420 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:11:59.191420 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:11:59.191420 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:11:59.191420 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:11:59.191420 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:11:59.191420 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:11:59.191420 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:11:59.191420 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:11:59.191420 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:11:59.191420 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:11:59.191420 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:11:59.191420 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:11:59.192289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:11:59.192289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:11:59.192289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:11:59.192289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:11:59.192289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:11:59.192289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:11:59.192289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:11:59.192289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:11:59.192289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:11:59.192289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:11:59.192289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:11:59.192289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:11:59.192289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:11:59.192289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:11:59.192289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:11:59.192289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:11:59.193159 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:11:59.193159 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:11:59.193159 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:11:59.193159 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:11:59.193159 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:11:59.193159 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:11:59.193159 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:11:59.193159 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:11:59.193159 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:11:59.193159 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:11:59.193159 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 15:11:59.193159 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:11:59.193159 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 53/104] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:11:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:11:59.206329 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6329851f-412b-4633-8c44-f9cae5b55d4e None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:59.311928 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:59.342673 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3648b2-fb6e-42e9-bf2d-6b9d99cf515e None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:59.347045 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6329851f-412b-4633-8c44-f9cae5b55d4e None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:59.387195 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2311e1d1-bef7-401f-852b-6acc21b8f063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:59.388003 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311e1d1-bef7-401f-852b-6acc21b8f063 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:59.388449 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311e1d1-bef7-401f-852b-6acc21b8f063 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:59.388834 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311e1d1-bef7-401f-852b-6acc21b8f063 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:59.392018 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b666067-67bb-4450-809a-abc71945bacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:59.392707 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b666067-67bb-4450-809a-abc71945bacf None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:59.393000 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b666067-67bb-4450-809a-abc71945bacf None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:59.393286 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b666067-67bb-4450-809a-abc71945bacf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:59.394752 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b666067-67bb-4450-809a-abc71945bacf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:59.396311 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b666067-67bb-4450-809a-abc71945bacf None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:59.399011 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 52/105] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:11:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:11:59.400161 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b666067-67bb-4450-809a-abc71945bacf None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:59.400640 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2311e1d1-bef7-401f-852b-6acc21b8f063 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:11:59.406136 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-2311e1d1-bef7-401f-852b-6acc21b8f063 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 15:11:59.406136 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:11:59.406136 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:11:59.406136 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:11:59.406136 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:11:59.406136 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:11:59.406136 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:11:59.406136 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:11:59.406136 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:11:59.406136 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:11:59.406136 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:11:59.406136 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:11:59.406136 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:11:59.406136 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:11:59.406136 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:11:59.406136 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:11:59.407160 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:11:59.407160 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:11:59.407160 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:11:59.407160 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:11:59.407160 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:11:59.407160 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:11:59.407160 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:11:59.407160 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:11:59.407160 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:11:59.407160 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:11:59.407160 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:11:59.407160 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:11:59.407160 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:11:59.407160 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:11:59.407160 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:11:59.407160 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:11:59.408139 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:11:59.408139 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:11:59.408139 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:11:59.408139 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:11:59.408139 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:11:59.408139 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:11:59.408139 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:11:59.408139 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:11:59.408139 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:11:59.408139 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:11:59.408139 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 15:11:59.408139 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:11:59.415307 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 54/106] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:11:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:11:59.416774 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a20e94-44af-4c9c-a1ba-634295150178 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:59.417176 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a20e94-44af-4c9c-a1ba-634295150178 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:59.417570 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a20e94-44af-4c9c-a1ba-634295150178 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:59.418988 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 55/107] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:11:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:11:59.419850 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a20e94-44af-4c9c-a1ba-634295150178 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:59.484452 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:59.508456 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b666067-67bb-4450-809a-abc71945bacf None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:59.545704 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ca203-71f5-416d-8280-5d52bdd6de3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:59.546415 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ca203-71f5-416d-8280-5d52bdd6de3a None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:59.546781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ca203-71f5-416d-8280-5d52bdd6de3a None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:59.547145 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ca203-71f5-416d-8280-5d52bdd6de3a None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:59.547939 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48ca203-71f5-416d-8280-5d52bdd6de3a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:11:59.548349 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48ca203-71f5-416d-8280-5d52bdd6de3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:59.549958 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48ca203-71f5-416d-8280-5d52bdd6de3a None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:59.563955 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 53/108] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 15:11:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:11:59.563955 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ca203-71f5-416d-8280-5d52bdd6de3a None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:59.645792 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a20e94-44af-4c9c-a1ba-634295150178 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:59.680766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ca203-71f5-416d-8280-5d52bdd6de3a None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:59.689895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:59.695775 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189ab5d-80d1-4f6a-be55-ed33968d3bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:59.696517 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189ab5d-80d1-4f6a-be55-ed33968d3bac None admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:59.696967 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189ab5d-80d1-4f6a-be55-ed33968d3bac None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:59.697411 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189ab5d-80d1-4f6a-be55-ed33968d3bac None admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:59.702134 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2189ab5d-80d1-4f6a-be55-ed33968d3bac None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:59.702134 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2189ab5d-80d1-4f6a-be55-ed33968d3bac None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:59.723795 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903c7d9-26be-47ae-af12-0d5142d5318c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:59.724455 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903c7d9-26be-47ae-af12-0d5142d5318c None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:59.724750 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903c7d9-26be-47ae-af12-0d5142d5318c None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:59.725034 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903c7d9-26be-47ae-af12-0d5142d5318c None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:59.725843 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f903c7d9-26be-47ae-af12-0d5142d5318c None admin] RBAC: Adding query filter params (name=member) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:11:59.726153 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f903c7d9-26be-47ae-af12-0d5142d5318c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:11:59.728117 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f903c7d9-26be-47ae-af12-0d5142d5318c None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:11:59.734025 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:11:59.738291 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 54/109] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 15:11:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:11:59.743034 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1aba95-93e7-4f42-9f01-ef517ef83787 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:59.743034 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1aba95-93e7-4f42-9f01-ef517ef83787 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:59.743034 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1aba95-93e7-4f42-9f01-ef517ef83787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:11:59.759571 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-ed1aba95-93e7-4f42-9f01-ef517ef83787 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:11:59.760852 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-ed1aba95-93e7-4f42-9f01-ef517ef83787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:11:59.788293 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 56/110] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 5 15:11:59 2022] POST /identity/v3/projects => generated 302 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:11:59.790721 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189ab5d-80d1-4f6a-be55-ed33968d3bac None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:59.899869 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 55/111] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:11:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:11:59.899869 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1aba95-93e7-4f42-9f01-ef517ef83787 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:11:59.947889 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189ab5d-80d1-4f6a-be55-ed33968d3bac None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:11:59.991007 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad412b12-95d2-4aa7-a9a4-08f172d6f382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:11:59.991697 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad412b12-95d2-4aa7-a9a4-08f172d6f382 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:11:59.991994 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad412b12-95d2-4aa7-a9a4-08f172d6f382 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:11:59.992358 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad412b12-95d2-4aa7-a9a4-08f172d6f382 None admin] PATH_INFO: `/v3/users/5d9236f58acd40bcbaf379964e48e4a2` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:00.010442 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad412b12-95d2-4aa7-a9a4-08f172d6f382 None admin] RBAC: Authorizing `identity:get_user(user_id=5d9236f58acd40bcbaf379964e48e4a2)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:00.013502 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad412b12-95d2-4aa7-a9a4-08f172d6f382 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:00.017068 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 57/112] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:11:59 2022] GET /identity/v3/users/5d9236f58acd40bcbaf379964e48e4a2 => generated 289 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:00.017488 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea68d2-af2c-41b1-b456-7ab0148af716 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:00.017779 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea68d2-af2c-41b1-b456-7ab0148af716 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:00.018057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea68d2-af2c-41b1-b456-7ab0148af716 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:00.019304 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 58/113] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:00.020736 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea68d2-af2c-41b1-b456-7ab0148af716 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:00.032797 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1aba95-93e7-4f42-9f01-ef517ef83787 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:00.072018 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:00.072497 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:00.082784 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eafd4b-c3c5-434c-b6bd-d3b955fb7772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:00.086265 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eafd4b-c3c5-434c-b6bd-d3b955fb7772 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:00.086265 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eafd4b-c3c5-434c-b6bd-d3b955fb7772 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:00.086265 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eafd4b-c3c5-434c-b6bd-d3b955fb7772 None admin] PATH_INFO: `/v3/users/a6cf4078f01748e98571f285e560294e` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:00.088497 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eafd4b-c3c5-434c-b6bd-d3b955fb7772 None admin] RBAC: Authorizing `identity:get_user(user_id=a6cf4078f01748e98571f285e560294e)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:00.089989 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eafd4b-c3c5-434c-b6bd-d3b955fb7772 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:00.091637 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 56/114] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:11:59 2022] GET /identity/v3/users/a6cf4078f01748e98571f285e560294e => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:00.096742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eafd4b-c3c5-434c-b6bd-d3b955fb7772 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:00.154962 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea68d2-af2c-41b1-b456-7ab0148af716 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:00.202601 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6b0b5c-6a60-4a0a-87f7-fd7efae7407f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:00.203695 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b0b5c-6a60-4a0a-87f7-fd7efae7407f None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:00.203798 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eafd4b-c3c5-434c-b6bd-d3b955fb7772 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:00.204294 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b0b5c-6a60-4a0a-87f7-fd7efae7407f None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:00.204770 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b0b5c-6a60-4a0a-87f7-fd7efae7407f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:00.218095 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6b0b5c-6a60-4a0a-87f7-fd7efae7407f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:00.223019 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-5b6b0b5c-6a60-4a0a-87f7-fd7efae7407f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:12:00.223019 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:00.223019 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:00.223019 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:00.223019 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:00.223019 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:00.223019 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:00.223019 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:00.223019 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:00.223019 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:00.223019 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:00.223019 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:00.223019 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:00.223019 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:00.223019 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:00.223019 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:00.223867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:00.223867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:00.223867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:00.223867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:00.223867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:00.223867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:00.223867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:00.223867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:00.223867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:00.223867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:00.223867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:00.223867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:00.223867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:00.223867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:00.223867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:00.223867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:00.224672 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:00.224672 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:00.224672 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:00.224672 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:00.224672 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:00.224672 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:00.224672 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:00.224672 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:00.224672 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:00.224672 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:00.224672 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:12:00.224672 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:00.228975 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 59/115] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 15:12:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:00.230366 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde0dc8-8a5d-4e2c-a524-e7da2d62caca None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:00.230366 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde0dc8-8a5d-4e2c-a524-e7da2d62caca None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:00.230656 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde0dc8-8a5d-4e2c-a524-e7da2d62caca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:00.242467 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904428c3-3a55-41e3-9cb9-06ecf5c469c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:00.243499 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904428c3-3a55-41e3-9cb9-06ecf5c469c6 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:00.243802 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904428c3-3a55-41e3-9cb9-06ecf5c469c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:00.244108 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904428c3-3a55-41e3-9cb9-06ecf5c469c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:00.248323 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-efde0dc8-8a5d-4e2c-a524-e7da2d62caca None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:00.249264 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-efde0dc8-8a5d-4e2c-a524-e7da2d62caca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:00.314067 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:00.317828 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904428c3-3a55-41e3-9cb9-06ecf5c469c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:00.319336 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904428c3-3a55-41e3-9cb9-06ecf5c469c6 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:00.320809 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 60/116] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:00.321725 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde0dc8-8a5d-4e2c-a524-e7da2d62caca None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:00.358019 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:00.432909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde0dc8-8a5d-4e2c-a524-e7da2d62caca None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:00.442325 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 57/117] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:12:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 15:12:00.442325 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdcedc-a9c6-466e-a3ba-eddf9bbca209 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:00.442325 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdcedc-a9c6-466e-a3ba-eddf9bbca209 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:00.442325 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdcedc-a9c6-466e-a3ba-eddf9bbca209 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:00.442325 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 58/118] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:00.442699 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cdcedc-a9c6-466e-a3ba-eddf9bbca209 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:00.471013 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867ee050-3b57-4f48-8307-1fcf0d699ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:00.471656 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ee050-3b57-4f48-8307-1fcf0d699ccf None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:00.471939 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ee050-3b57-4f48-8307-1fcf0d699ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:00.472275 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ee050-3b57-4f48-8307-1fcf0d699ccf None admin] PATH_INFO: `/v3/projects/b972ca1ff8ee4c558eeb45ec24303061` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:00.473425 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867ee050-3b57-4f48-8307-1fcf0d699ccf None admin] RBAC: Authorizing `identity:get_project(project_id=b972ca1ff8ee4c558eeb45ec24303061)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:00.474872 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867ee050-3b57-4f48-8307-1fcf0d699ccf None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:00.476615 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 61/119] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:12:00 2022] GET /identity/v3/projects/b972ca1ff8ee4c558eeb45ec24303061 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:00.477361 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867ee050-3b57-4f48-8307-1fcf0d699ccf None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:00.546460 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:00.551134 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cdcedc-a9c6-466e-a3ba-eddf9bbca209 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:00.579070 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:00.582216 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867ee050-3b57-4f48-8307-1fcf0d699ccf None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:00.586656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d337657-57b1-4aef-bca3-eadccfc3585a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:00.587280 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d337657-57b1-4aef-bca3-eadccfc3585a None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:00.587566 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d337657-57b1-4aef-bca3-eadccfc3585a None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:00.587845 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d337657-57b1-4aef-bca3-eadccfc3585a None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:00.588920 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d337657-57b1-4aef-bca3-eadccfc3585a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:00.588920 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d337657-57b1-4aef-bca3-eadccfc3585a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:00.590317 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d337657-57b1-4aef-bca3-eadccfc3585a None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:00.604365 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 59/120] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:12:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:00.604365 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d337657-57b1-4aef-bca3-eadccfc3585a None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:00.623573 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b70ea2-8ac0-4fdb-b9f0-707470711b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:00.624571 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b70ea2-8ac0-4fdb-b9f0-707470711b1b None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:00.625353 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b70ea2-8ac0-4fdb-b9f0-707470711b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:00.626450 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b70ea2-8ac0-4fdb-b9f0-707470711b1b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:00.638249 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b70ea2-8ac0-4fdb-b9f0-707470711b1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:00.643168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-63b70ea2-8ac0-4fdb-b9f0-707470711b1b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 15:12:00.643168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:00.643168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:00.643168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:00.643168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:00.643168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:00.643168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:00.643168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:00.643168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:00.643168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:00.643168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:00.643168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:00.643168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:00.643168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:00.643168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:00.643168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:00.644013 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:00.644013 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:00.644013 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:00.644013 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:00.644013 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:00.644013 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:00.644013 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:00.644013 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:00.644013 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:00.644013 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:00.644013 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:00.644013 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:00.644013 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:00.644013 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:00.644013 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:00.644013 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:00.650068 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:00.650068 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:00.650068 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:00.650068 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:00.650068 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:00.650068 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:00.650068 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:00.650068 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:00.650068 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:00.650068 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:00.650068 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 15:12:00.650068 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:00.650068 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 62/121] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 15:12:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:00.650068 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae60e4-b4fa-4b14-90ae-6d92e70e8657 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:00.650767 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae60e4-b4fa-4b14-90ae-6d92e70e8657 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:00.650767 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae60e4-b4fa-4b14-90ae-6d92e70e8657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:00.668375 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-82ae60e4-b4fa-4b14-90ae-6d92e70e8657 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:00.669613 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-82ae60e4-b4fa-4b14-90ae-6d92e70e8657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:00.738914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d337657-57b1-4aef-bca3-eadccfc3585a None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:00.748242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 63/122] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:00.748242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae60e4-b4fa-4b14-90ae-6d92e70e8657 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:00.790218 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50055f8e-41e3-4b7d-9c4b-e17859424b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:00.790872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50055f8e-41e3-4b7d-9c4b-e17859424b8e None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:00.791162 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50055f8e-41e3-4b7d-9c4b-e17859424b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:00.791447 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50055f8e-41e3-4b7d-9c4b-e17859424b8e None admin] PATH_INFO: `/v3/projects/b972ca1ff8ee4c558eeb45ec24303061` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:00.792659 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50055f8e-41e3-4b7d-9c4b-e17859424b8e None admin] RBAC: Authorizing `identity:get_project(project_id=b972ca1ff8ee4c558eeb45ec24303061)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:00.794188 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50055f8e-41e3-4b7d-9c4b-e17859424b8e None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:00.800852 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 60/123] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:12:00 2022] GET /identity/v3/projects/b972ca1ff8ee4c558eeb45ec24303061 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:00.800852 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50055f8e-41e3-4b7d-9c4b-e17859424b8e None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:00.827770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:00.840617 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:00.850297 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae60e4-b4fa-4b14-90ae-6d92e70e8657 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:00.886221 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756bc8d-3d61-4ee2-b263-6d31cdb95981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:00.887024 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756bc8d-3d61-4ee2-b263-6d31cdb95981 None admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:00.887412 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756bc8d-3d61-4ee2-b263-6d31cdb95981 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:00.887787 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756bc8d-3d61-4ee2-b263-6d31cdb95981 None admin] PATH_INFO: `/v3/projects/b972ca1ff8ee4c558eeb45ec24303061/users/a6cf4078f01748e98571f285e560294e/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:00.889866 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2756bc8d-3d61-4ee2-b263-6d31cdb95981 None admin] RBAC: Authorizing `identity:create_grant(project_id=b972ca1ff8ee4c558eeb45ec24303061, user_id=a6cf4078f01748e98571f285e560294e, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:00.891420 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2756bc8d-3d61-4ee2-b263-6d31cdb95981 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:00.902458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50055f8e-41e3-4b7d-9c4b-e17859424b8e None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:00.936997 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c28fa-29f3-494d-9b8b-10dfa8041ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:00.937643 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c28fa-29f3-494d-9b8b-10dfa8041ce9 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:00.937933 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c28fa-29f3-494d-9b8b-10dfa8041ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:00.938218 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c28fa-29f3-494d-9b8b-10dfa8041ce9 None admin] PATH_INFO: `/v3/users/338271833df8486b9ef50964b5ec7797` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:00.939396 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340c28fa-29f3-494d-9b8b-10dfa8041ce9 None admin] RBAC: Authorizing `identity:get_user(user_id=338271833df8486b9ef50964b5ec7797)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:00.940883 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340c28fa-29f3-494d-9b8b-10dfa8041ce9 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:00.942489 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 61/124] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:12:00 2022] GET /identity/v3/users/338271833df8486b9ef50964b5ec7797 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:00.944968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c28fa-29f3-494d-9b8b-10dfa8041ce9 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:00.972839 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 64/125] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 15:12:00 2022] PUT /identity/v3/projects/b972ca1ff8ee4c558eeb45ec24303061/users/a6cf4078f01748e98571f285e560294e/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:12:00.973614 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756bc8d-3d61-4ee2-b263-6d31cdb95981 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:01.069590 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:01.077602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c28fa-29f3-494d-9b8b-10dfa8041ce9 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:01.109745 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756bc8d-3d61-4ee2-b263-6d31cdb95981 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:01.118324 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f032c836-fb62-47c9-8c3b-52e8cfabe0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:01.119293 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f032c836-fb62-47c9-8c3b-52e8cfabe0dc None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:01.119842 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f032c836-fb62-47c9-8c3b-52e8cfabe0dc None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:01.120378 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f032c836-fb62-47c9-8c3b-52e8cfabe0dc None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:01.121390 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f032c836-fb62-47c9-8c3b-52e8cfabe0dc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:01.121921 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f032c836-fb62-47c9-8c3b-52e8cfabe0dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:01.123499 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f032c836-fb62-47c9-8c3b-52e8cfabe0dc None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:01.133072 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 62/126] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:12:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:01.134505 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f032c836-fb62-47c9-8c3b-52e8cfabe0dc None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:01.155026 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd6098-2c15-457f-a9d3-2163ca55d77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:01.155685 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd6098-2c15-457f-a9d3-2163ca55d77f None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:01.155977 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd6098-2c15-457f-a9d3-2163ca55d77f None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:01.156311 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd6098-2c15-457f-a9d3-2163ca55d77f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:01.169853 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fd6098-2c15-457f-a9d3-2163ca55d77f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:01.174810 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-00fd6098-2c15-457f-a9d3-2163ca55d77f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 15:12:01.174810 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:01.174810 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:01.174810 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:01.174810 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:01.174810 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:01.174810 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:01.174810 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:01.174810 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:01.174810 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:01.174810 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:01.174810 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:01.174810 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:01.174810 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:01.174810 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:01.174810 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:01.175935 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:01.175935 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:01.175935 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:01.175935 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:01.175935 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:01.175935 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:01.175935 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:01.175935 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:01.175935 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:01.175935 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:01.175935 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:01.175935 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:01.175935 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:01.175935 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:01.175935 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:01.175935 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:01.176895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:01.176895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:01.176895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:01.176895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:01.176895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:01.176895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:01.176895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:01.176895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:01.176895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:01.176895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:01.176895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 15:12:01.176895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:01.176895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 65/127] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:12:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:01.178286 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd6098-2c15-457f-a9d3-2163ca55d77f None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:01.193068 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:01.281075 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f032c836-fb62-47c9-8c3b-52e8cfabe0dc None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:01.290749 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd6098-2c15-457f-a9d3-2163ca55d77f None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:01.326150 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a1f4da-aa90-4421-8aa7-9a4c45deab3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:01.326705 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1f4da-aa90-4421-8aa7-9a4c45deab3b None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:01.326993 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1f4da-aa90-4421-8aa7-9a4c45deab3b None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:01.327274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1f4da-aa90-4421-8aa7-9a4c45deab3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:01.328115 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a1f4da-aa90-4421-8aa7-9a4c45deab3b None admin] RBAC: Adding query filter params (role.id=09be81c3953242a08b57fd702a5082e3, scope.project.id=b972ca1ff8ee4c558eeb45ec24303061, user.id=5d9236f58acd40bcbaf379964e48e4a2) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:01.328453 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a1f4da-aa90-4421-8aa7-9a4c45deab3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:01.329922 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a1f4da-aa90-4421-8aa7-9a4c45deab3b None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:01.336243 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325b939e-9a2f-4630-85da-c6d3c4828580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:01.336691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 63/128] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:12:01 2022] GET /identity/v3/role_assignments?user.id=5d9236f58acd40bcbaf379964e48e4a2&scope.project.id=b972ca1ff8ee4c558eeb45ec24303061&role.id=09be81c3953242a08b57fd702a5082e3 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:01.337442 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b939e-9a2f-4630-85da-c6d3c4828580 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:01.337565 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a1f4da-aa90-4421-8aa7-9a4c45deab3b None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:01.338132 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b939e-9a2f-4630-85da-c6d3c4828580 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:01.338639 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b939e-9a2f-4630-85da-c6d3c4828580 None admin] PATH_INFO: `/v3/projects/b972ca1ff8ee4c558eeb45ec24303061` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:01.340478 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325b939e-9a2f-4630-85da-c6d3c4828580 None admin] RBAC: Authorizing `identity:get_project(project_id=b972ca1ff8ee4c558eeb45ec24303061)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:01.343336 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325b939e-9a2f-4630-85da-c6d3c4828580 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:01.345557 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 66/129] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:12:01 2022] GET /identity/v3/projects/b972ca1ff8ee4c558eeb45ec24303061 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:01.351671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325b939e-9a2f-4630-85da-c6d3c4828580 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:01.376553 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:01.473434 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a1f4da-aa90-4421-8aa7-9a4c45deab3b None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:01.515996 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050eacdd-d559-4a49-a1af-5eb69149cb51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:01.517093 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050eacdd-d559-4a49-a1af-5eb69149cb51 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:01.517586 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050eacdd-d559-4a49-a1af-5eb69149cb51 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:01.518298 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050eacdd-d559-4a49-a1af-5eb69149cb51 None admin] PATH_INFO: `/v3/users/a6cf4078f01748e98571f285e560294e` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:01.519855 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050eacdd-d559-4a49-a1af-5eb69149cb51 None admin] RBAC: Authorizing `identity:get_user(user_id=a6cf4078f01748e98571f285e560294e)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:01.522263 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050eacdd-d559-4a49-a1af-5eb69149cb51 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:01.525195 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 64/130] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:12:01 2022] GET /identity/v3/users/a6cf4078f01748e98571f285e560294e => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:01.526182 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050eacdd-d559-4a49-a1af-5eb69149cb51 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:01.575141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:01.613687 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325b939e-9a2f-4630-85da-c6d3c4828580 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:01.655893 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2824f952-550a-4bb3-a015-aea1f11009bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:01.656557 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2824f952-550a-4bb3-a015-aea1f11009bb None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:01.656850 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2824f952-550a-4bb3-a015-aea1f11009bb None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:01.657135 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2824f952-550a-4bb3-a015-aea1f11009bb None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:01.657864 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2824f952-550a-4bb3-a015-aea1f11009bb None admin] RBAC: Adding query filter params (name=member) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:01.658181 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2824f952-550a-4bb3-a015-aea1f11009bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:01.659574 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2824f952-550a-4bb3-a015-aea1f11009bb None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:01.682052 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 67/131] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 15:12:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:01.682932 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2824f952-550a-4bb3-a015-aea1f11009bb None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:01.695953 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:01.716357 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050eacdd-d559-4a49-a1af-5eb69149cb51 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:01.812257 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd01866-7e62-4d17-9919-eb6618ba68b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:01.812966 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd01866-7e62-4d17-9919-eb6618ba68b8 None admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:01.813257 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd01866-7e62-4d17-9919-eb6618ba68b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:01.813545 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd01866-7e62-4d17-9919-eb6618ba68b8 None admin] PATH_INFO: `/v3/projects/b972ca1ff8ee4c558eeb45ec24303061/users/338271833df8486b9ef50964b5ec7797/roles/7c88c815db854954b516ab97cd471647` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:01.816703 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2824f952-550a-4bb3-a015-aea1f11009bb None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:01.821644 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd01866-7e62-4d17-9919-eb6618ba68b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=b972ca1ff8ee4c558eeb45ec24303061, user_id=338271833df8486b9ef50964b5ec7797, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:01.823139 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd01866-7e62-4d17-9919-eb6618ba68b8 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:01.854496 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522df70a-7521-4d59-8409-f39a2dcb3c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:01.855156 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522df70a-7521-4d59-8409-f39a2dcb3c15 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:01.855452 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522df70a-7521-4d59-8409-f39a2dcb3c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:01.855736 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522df70a-7521-4d59-8409-f39a2dcb3c15 None admin] PATH_INFO: `/v3/projects/b972ca1ff8ee4c558eeb45ec24303061` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:01.857035 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522df70a-7521-4d59-8409-f39a2dcb3c15 None admin] RBAC: Authorizing `identity:get_project(project_id=b972ca1ff8ee4c558eeb45ec24303061)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:01.858544 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522df70a-7521-4d59-8409-f39a2dcb3c15 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:01.860310 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 68/132] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:12:01 2022] GET /identity/v3/projects/b972ca1ff8ee4c558eeb45ec24303061 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:01.861238 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522df70a-7521-4d59-8409-f39a2dcb3c15 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:01.874855 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:01.930857 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 65/133] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 15:12:01 2022] PUT /identity/v3/projects/b972ca1ff8ee4c558eeb45ec24303061/users/338271833df8486b9ef50964b5ec7797/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:12:01.931674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd01866-7e62-4d17-9919-eb6618ba68b8 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:01.972384 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522df70a-7521-4d59-8409-f39a2dcb3c15 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:02.013648 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151a7bca-0e96-48f6-b0d5-632d5bf4fc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:02.014414 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a7bca-0e96-48f6-b0d5-632d5bf4fc1d None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:02.014716 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a7bca-0e96-48f6-b0d5-632d5bf4fc1d None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:02.015006 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a7bca-0e96-48f6-b0d5-632d5bf4fc1d None admin] PATH_INFO: `/v3/users/a6cf4078f01748e98571f285e560294e` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:02.016368 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151a7bca-0e96-48f6-b0d5-632d5bf4fc1d None admin] RBAC: Authorizing `identity:get_user(user_id=a6cf4078f01748e98571f285e560294e)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:02.018234 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151a7bca-0e96-48f6-b0d5-632d5bf4fc1d None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:02.020126 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 69/134] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:12:01 2022] GET /identity/v3/users/a6cf4078f01748e98571f285e560294e => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:02.021637 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096510e-fe5c-4ba9-b5b7-a96b3f28daf7 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:02.022042 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096510e-fe5c-4ba9-b5b7-a96b3f28daf7 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:02.022336 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096510e-fe5c-4ba9-b5b7-a96b3f28daf7 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:02.023774 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 70/135] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:02.028997 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b096510e-fe5c-4ba9-b5b7-a96b3f28daf7 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:02.062759 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd01866-7e62-4d17-9919-eb6618ba68b8 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:02.101663 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26292091-4d9d-40bd-9b92-1eab5e7dc9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:02.102363 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26292091-4d9d-40bd-9b92-1eab5e7dc9a4 None admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:02.102656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26292091-4d9d-40bd-9b92-1eab5e7dc9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:02.102995 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26292091-4d9d-40bd-9b92-1eab5e7dc9a4 None admin] PATH_INFO: `/v3/projects/b972ca1ff8ee4c558eeb45ec24303061/users/a6cf4078f01748e98571f285e560294e/roles/944427ca14ee4c81a05ec67e54f50e7f` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:02.105384 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26292091-4d9d-40bd-9b92-1eab5e7dc9a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=b972ca1ff8ee4c558eeb45ec24303061, user_id=a6cf4078f01748e98571f285e560294e, role_id=944427ca14ee4c81a05ec67e54f50e7f)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:02.106886 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26292091-4d9d-40bd-9b92-1eab5e7dc9a4 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:02.129253 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:02.155091 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b096510e-fe5c-4ba9-b5b7-a96b3f28daf7 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:02.192264 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 66/136] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 15:12:01 2022] PUT /identity/v3/projects/b972ca1ff8ee4c558eeb45ec24303061/users/a6cf4078f01748e98571f285e560294e/roles/944427ca14ee4c81a05ec67e54f50e7f => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:12:02.192456 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7617e-258c-42b3-83c3-42618023a207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:02.193068 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7617e-258c-42b3-83c3-42618023a207 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:02.193354 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7617e-258c-42b3-83c3-42618023a207 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:02.193633 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7617e-258c-42b3-83c3-42618023a207 None admin] PATH_INFO: `/v3/projects/eab4ef6110f1406cbd92b046713d575b` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:02.193833 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e6e0f-3f5b-4df2-ae6a-aea3beb2ce28 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:02.194848 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7617e-258c-42b3-83c3-42618023a207 None admin] RBAC: Authorizing `identity:get_project(project_id=eab4ef6110f1406cbd92b046713d575b)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:02.196344 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7617e-258c-42b3-83c3-42618023a207 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:02.198409 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 71/137] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:12:02 2022] GET /identity/v3/projects/eab4ef6110f1406cbd92b046713d575b => generated 312 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:02.198730 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e6e0f-3f5b-4df2-ae6a-aea3beb2ce28 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:02.199092 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e6e0f-3f5b-4df2-ae6a-aea3beb2ce28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:02.204796 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7617e-258c-42b3-83c3-42618023a207 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:02.235626 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-a02e6e0f-3f5b-4df2-ae6a-aea3beb2ce28 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:02.238912 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-a02e6e0f-3f5b-4df2-ae6a-aea3beb2ce28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:02.247435 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:02.391789 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 67/138] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:02.407211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7617e-258c-42b3-83c3-42618023a207 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:02.443757 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a9239e-fc3a-4244-9679-186dc76a9ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:02.444788 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9239e-fc3a-4244-9679-186dc76a9ade None admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:02.445247 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9239e-fc3a-4244-9679-186dc76a9ade None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:02.445723 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9239e-fc3a-4244-9679-186dc76a9ade None admin] PATH_INFO: `/v3/projects/eab4ef6110f1406cbd92b046713d575b/users/a6cf4078f01748e98571f285e560294e/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:02.447763 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e6e0f-3f5b-4df2-ae6a-aea3beb2ce28 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:02.456289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a9239e-fc3a-4244-9679-186dc76a9ade None admin] RBAC: Authorizing `identity:create_grant(project_id=eab4ef6110f1406cbd92b046713d575b, user_id=a6cf4078f01748e98571f285e560294e, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:02.457981 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a9239e-fc3a-4244-9679-186dc76a9ade None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:02.515498 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 72/139] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 15:12:02 2022] PUT /identity/v3/projects/eab4ef6110f1406cbd92b046713d575b/users/a6cf4078f01748e98571f285e560294e/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:12:02.571653 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e6e0f-3f5b-4df2-ae6a-aea3beb2ce28 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:02.611078 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d5e2ba-e5af-4437-9ab2-2fe06f6702c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:02.612023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d5e2ba-e5af-4437-9ab2-2fe06f6702c4 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:02.612503 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d5e2ba-e5af-4437-9ab2-2fe06f6702c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:02.612945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d5e2ba-e5af-4437-9ab2-2fe06f6702c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:02.614350 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d5e2ba-e5af-4437-9ab2-2fe06f6702c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:02.616181 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d5e2ba-e5af-4437-9ab2-2fe06f6702c4 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:02.618064 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 68/140] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:12:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:12:02.623631 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a9239e-fc3a-4244-9679-186dc76a9ade None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:02.629922 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:02.749602 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a9239e-fc3a-4244-9679-186dc76a9ade None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:02.784996 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e77a9-75c3-4958-9a92-a2eb97ffa54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:02.785617 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e77a9-75c3-4958-9a92-a2eb97ffa54b None admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:02.785906 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e77a9-75c3-4958-9a92-a2eb97ffa54b None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:02.786187 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e77a9-75c3-4958-9a92-a2eb97ffa54b None admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:02.786881 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84e77a9-75c3-4958-9a92-a2eb97ffa54b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:02.788397 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84e77a9-75c3-4958-9a92-a2eb97ffa54b None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:02.905679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 73/141] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 5 15:12:02 2022] POST /identity/v3/users => generated 283 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:12:03.626128 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e79a80-622e-4df1-ba8a-4a3f2a5210d8 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:03.626632 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e79a80-622e-4df1-ba8a-4a3f2a5210d8 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:03.626735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e79a80-622e-4df1-ba8a-4a3f2a5210d8 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:03.628196 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 69/142] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:03.634863 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ce909-a2d4-4943-942f-ad8e0e3cfcc4 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:03.635287 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ce909-a2d4-4943-942f-ad8e0e3cfcc4 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:03.635475 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ce909-a2d4-4943-942f-ad8e0e3cfcc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:03.654140 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-989ce909-a2d4-4943-942f-ad8e0e3cfcc4 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:03.655350 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-989ce909-a2d4-4943-942f-ad8e0e3cfcc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:03.718106 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:03.729769 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 74/143] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:03.753954 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e79a80-622e-4df1-ba8a-4a3f2a5210d8 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:03.766304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:03.864387 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e79a80-622e-4df1-ba8a-4a3f2a5210d8 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:03.900589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc2834-1888-47b1-9d44-43259e9cf68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:03.901158 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc2834-1888-47b1-9d44-43259e9cf68f None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:03.901443 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc2834-1888-47b1-9d44-43259e9cf68f None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:03.901724 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc2834-1888-47b1-9d44-43259e9cf68f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:03.912410 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cc2834-1888-47b1-9d44-43259e9cf68f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:03.917924 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-22cc2834-1888-47b1-9d44-43259e9cf68f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 15:12:03.917924 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:03.917924 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:03.917924 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:03.917924 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:03.917924 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:03.917924 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:03.917924 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:03.917924 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:03.917924 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:03.917924 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:03.917924 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:03.917924 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:03.917924 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:03.917924 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:03.917924 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:03.918762 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:03.918762 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:03.918762 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:03.918762 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:03.918762 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:03.918762 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:03.918762 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:03.918762 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:03.918762 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:03.918762 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:03.918762 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:03.918762 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:03.918762 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:03.918762 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:03.918762 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:03.918762 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:03.919535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:03.919535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:03.919535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:03.919535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:03.919535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:03.919535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:03.919535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:03.919535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:03.919535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:03.919535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:03.919535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 15:12:03.919535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:03.919535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 70/144] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:12:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:03.925200 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ce909-a2d4-4943-942f-ad8e0e3cfcc4 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:04.029607 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ce909-a2d4-4943-942f-ad8e0e3cfcc4 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:04.069552 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d10ef-fbf6-48b1-94f6-9c161f365dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:04.070209 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d10ef-fbf6-48b1-94f6-9c161f365dde None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:04.070493 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d10ef-fbf6-48b1-94f6-9c161f365dde None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:04.070792 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d10ef-fbf6-48b1-94f6-9c161f365dde None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:04.071489 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d10ef-fbf6-48b1-94f6-9c161f365dde None admin] RBAC: Adding query filter params (name=member) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:04.071799 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d10ef-fbf6-48b1-94f6-9c161f365dde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:04.073203 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d10ef-fbf6-48b1-94f6-9c161f365dde None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:04.087031 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 75/145] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 15:12:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:12:04.092635 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc2834-1888-47b1-9d44-43259e9cf68f None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:04.103399 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32b20d-9b57-4f6c-9989-528148f6a571 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:04.103831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32b20d-9b57-4f6c-9989-528148f6a571 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:04.104259 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32b20d-9b57-4f6c-9989-528148f6a571 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:04.106010 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 76/146] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:04.111579 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad3e24-bd97-46c9-be5f-36aa351f5e7f None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:04.111881 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad3e24-bd97-46c9-be5f-36aa351f5e7f None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:04.112227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad3e24-bd97-46c9-be5f-36aa351f5e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:04.140270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-e0ad3e24-bd97-46c9-be5f-36aa351f5e7f None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:04.141516 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-e0ad3e24-bd97-46c9-be5f-36aa351f5e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:04.173881 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:04.212379 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc2834-1888-47b1-9d44-43259e9cf68f None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:04.252102 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4a8bf6-2b70-4eca-81d2-c0e57351863e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:04.252776 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a8bf6-2b70-4eca-81d2-c0e57351863e None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:04.252882 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:04.253392 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a8bf6-2b70-4eca-81d2-c0e57351863e None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:04.253707 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a8bf6-2b70-4eca-81d2-c0e57351863e None admin] PATH_INFO: `/v3/users/a6cf4078f01748e98571f285e560294e` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:04.254934 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4a8bf6-2b70-4eca-81d2-c0e57351863e None admin] RBAC: Authorizing `identity:get_user(user_id=a6cf4078f01748e98571f285e560294e)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:04.256577 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4a8bf6-2b70-4eca-81d2-c0e57351863e None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:04.258231 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 71/147] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:12:04 2022] GET /identity/v3/users/a6cf4078f01748e98571f285e560294e => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:04.264479 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4a8bf6-2b70-4eca-81d2-c0e57351863e None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:04.293935 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 77/148] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 15:12:04.331857 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ad3e24-bd97-46c9-be5f-36aa351f5e7f None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:04.379465 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4a8bf6-2b70-4eca-81d2-c0e57351863e None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:04.425350 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c05e422-aefc-4eba-b3fd-9dd724015d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:04.425905 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05e422-aefc-4eba-b3fd-9dd724015d75 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:04.426185 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05e422-aefc-4eba-b3fd-9dd724015d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:04.426458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05e422-aefc-4eba-b3fd-9dd724015d75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:04.432388 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:04.529657 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c05e422-aefc-4eba-b3fd-9dd724015d75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:04.530384 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ad3e24-bd97-46c9-be5f-36aa351f5e7f None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:04.531166 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c05e422-aefc-4eba-b3fd-9dd724015d75 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:04.575140 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb1877c-7d67-4e62-9a06-eea1c1d0dcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:04.575792 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1877c-7d67-4e62-9a06-eea1c1d0dcbe None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:04.576114 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1877c-7d67-4e62-9a06-eea1c1d0dcbe None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:04.576449 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1877c-7d67-4e62-9a06-eea1c1d0dcbe None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:04.590027 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb1877c-7d67-4e62-9a06-eea1c1d0dcbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:04.595083 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-0fb1877c-7d67-4e62-9a06-eea1c1d0dcbe None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 15:12:04.595083 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:04.595083 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:04.595083 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:04.595083 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:04.595083 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:04.595083 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:04.595083 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:04.595083 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:04.595083 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:04.595083 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:04.595083 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:04.595083 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:04.595083 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:04.595083 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:04.595083 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:04.595946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:04.595946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:04.595946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:04.595946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:04.595946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:04.595946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:04.595946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:04.595946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:04.595946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:04.595946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:04.595946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:04.595946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:04.595946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:04.595946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:04.595946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:04.595946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:04.596738 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:04.596738 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:04.596738 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:04.596738 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:04.596738 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:04.596738 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:04.596738 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:04.596738 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:04.596738 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:04.596738 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:04.596738 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 15:12:04.596738 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:04.596738 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 78/149] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:12:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:04.602483 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb1877c-7d67-4e62-9a06-eea1c1d0dcbe None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:04.651574 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:04.653704 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 72/150] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:12:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 15:12:04.659747 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c05e422-aefc-4eba-b3fd-9dd724015d75 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:04.716532 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:04.718717 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb1877c-7d67-4e62-9a06-eea1c1d0dcbe None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:04.757979 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532863e1-7067-4a64-89c1-ea6d813f0fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:04.758663 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532863e1-7067-4a64-89c1-ea6d813f0fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:04.758950 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532863e1-7067-4a64-89c1-ea6d813f0fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:04.759225 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532863e1-7067-4a64-89c1-ea6d813f0fb7 None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:04.759999 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532863e1-7067-4a64-89c1-ea6d813f0fb7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:04.760335 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532863e1-7067-4a64-89c1-ea6d813f0fb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:04.761723 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532863e1-7067-4a64-89c1-ea6d813f0fb7 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:04.770627 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 79/151] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 15:12:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:12:04.774525 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c05e422-aefc-4eba-b3fd-9dd724015d75 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:04.776655 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532863e1-7067-4a64-89c1-ea6d813f0fb7 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:04.816408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd84a774-1f42-487a-8415-01d4a396a30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:04.817074 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd84a774-1f42-487a-8415-01d4a396a30f None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:04.817354 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd84a774-1f42-487a-8415-01d4a396a30f None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:04.817630 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd84a774-1f42-487a-8415-01d4a396a30f None admin] PATH_INFO: `/v3/projects/b972ca1ff8ee4c558eeb45ec24303061` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:04.820927 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd84a774-1f42-487a-8415-01d4a396a30f None admin] RBAC: Authorizing `identity:get_project(project_id=b972ca1ff8ee4c558eeb45ec24303061)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:04.822386 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd84a774-1f42-487a-8415-01d4a396a30f None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:04.824093 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 73/152] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:12:04 2022] GET /identity/v3/projects/b972ca1ff8ee4c558eeb45ec24303061 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:12:04.829923 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd84a774-1f42-487a-8415-01d4a396a30f None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:04.890025 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532863e1-7067-4a64-89c1-ea6d813f0fb7 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:04.892559 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:04.925268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6264a5ea-d80e-4e3d-9f61-1021a86fa929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:04.926254 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264a5ea-d80e-4e3d-9f61-1021a86fa929 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:04.926712 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264a5ea-d80e-4e3d-9f61-1021a86fa929 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:04.927204 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264a5ea-d80e-4e3d-9f61-1021a86fa929 None admin] PATH_INFO: `/v3/users/5d9236f58acd40bcbaf379964e48e4a2` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:04.928452 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6264a5ea-d80e-4e3d-9f61-1021a86fa929 None admin] RBAC: Authorizing `identity:get_user(user_id=5d9236f58acd40bcbaf379964e48e4a2)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:04.930574 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6264a5ea-d80e-4e3d-9f61-1021a86fa929 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:04.932265 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 80/153] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:12:04 2022] GET /identity/v3/users/5d9236f58acd40bcbaf379964e48e4a2 => generated 289 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:12:04.939380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6264a5ea-d80e-4e3d-9f61-1021a86fa929 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:04.978584 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd84a774-1f42-487a-8415-01d4a396a30f None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:04.996661 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:05.040785 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9078cd6-f569-4dd7-963b-786d743c9fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:05.041425 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9078cd6-f569-4dd7-963b-786d743c9fbd None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:05.041714 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9078cd6-f569-4dd7-963b-786d743c9fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:05.042046 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9078cd6-f569-4dd7-963b-786d743c9fbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:05.042867 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9078cd6-f569-4dd7-963b-786d743c9fbd None admin] RBAC: Adding query filter params (role.id=d6c91838e4944cfbb23e1b6ecb009d26, scope.project.id=b972ca1ff8ee4c558eeb45ec24303061, user.id=a6cf4078f01748e98571f285e560294e) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:05.043782 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9078cd6-f569-4dd7-963b-786d743c9fbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:05.044668 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9078cd6-f569-4dd7-963b-786d743c9fbd None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:05.052117 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 74/154] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:12:04 2022] GET /identity/v3/role_assignments?user.id=a6cf4078f01748e98571f285e560294e&scope.project.id=b972ca1ff8ee4c558eeb45ec24303061&role.id=d6c91838e4944cfbb23e1b6ecb009d26 => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:05.092395 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6264a5ea-d80e-4e3d-9f61-1021a86fa929 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:05.140140 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2203172b-fb87-49a8-9aa7-20e211b9b17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:05.141032 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203172b-fb87-49a8-9aa7-20e211b9b17c None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:05.141513 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203172b-fb87-49a8-9aa7-20e211b9b17c None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:05.141967 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203172b-fb87-49a8-9aa7-20e211b9b17c None admin] PATH_INFO: `/v3/projects/b972ca1ff8ee4c558eeb45ec24303061` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:05.143284 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2203172b-fb87-49a8-9aa7-20e211b9b17c None admin] RBAC: Authorizing `identity:get_project(project_id=b972ca1ff8ee4c558eeb45ec24303061)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:05.144972 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2203172b-fb87-49a8-9aa7-20e211b9b17c None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:05.146891 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 81/155] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:12:04 2022] GET /identity/v3/projects/b972ca1ff8ee4c558eeb45ec24303061 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:05.154639 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9078cd6-f569-4dd7-963b-786d743c9fbd None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:05.309278 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9078cd6-f569-4dd7-963b-786d743c9fbd None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:05.345562 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66adba4-1237-417b-8fc1-c8a1a9d7d8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:05.346250 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66adba4-1237-417b-8fc1-c8a1a9d7d8f5 None admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:05.346533 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66adba4-1237-417b-8fc1-c8a1a9d7d8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:05.346811 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66adba4-1237-417b-8fc1-c8a1a9d7d8f5 None admin] PATH_INFO: `/v3/projects/b972ca1ff8ee4c558eeb45ec24303061/users/5d9236f58acd40bcbaf379964e48e4a2/roles/09be81c3953242a08b57fd702a5082e3` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:05.352859 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66adba4-1237-417b-8fc1-c8a1a9d7d8f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=b972ca1ff8ee4c558eeb45ec24303061, user_id=5d9236f58acd40bcbaf379964e48e4a2, role_id=09be81c3953242a08b57fd702a5082e3)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:05.354425 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66adba4-1237-417b-8fc1-c8a1a9d7d8f5 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:05.443007 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 75/156] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 15:12:05 2022] PUT /identity/v3/projects/b972ca1ff8ee4c558eeb45ec24303061/users/5d9236f58acd40bcbaf379964e48e4a2/roles/09be81c3953242a08b57fd702a5082e3 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:12:05.763701 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40fe10-2e45-4ced-9d27-3a1d8639edfc None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:05.764251 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40fe10-2e45-4ced-9d27-3a1d8639edfc None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:05.764325 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40fe10-2e45-4ced-9d27-3a1d8639edfc None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:05.765920 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 82/157] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:05.774530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b3fce-b8d1-4857-807f-10f619615ce2 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:05.775034 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b3fce-b8d1-4857-807f-10f619615ce2 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:05.775110 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b3fce-b8d1-4857-807f-10f619615ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:05.801123 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-334b3fce-b8d1-4857-807f-10f619615ce2 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:05.802338 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-334b3fce-b8d1-4857-807f-10f619615ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:05.808499 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd79d0-30d9-429a-b65f-a893405172b0 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:05.808784 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd79d0-30d9-429a-b65f-a893405172b0 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:05.809068 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd79d0-30d9-429a-b65f-a893405172b0 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:05.810499 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 83/158] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 15:12:05.815663 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab33523-306f-4462-b52e-110a567cb1e4 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:05.815946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab33523-306f-4462-b52e-110a567cb1e4 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:05.816260 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab33523-306f-4462-b52e-110a567cb1e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:05.852367 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-fab33523-306f-4462-b52e-110a567cb1e4 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:05.853579 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-fab33523-306f-4462-b52e-110a567cb1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:05.864425 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:05.932186 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:05.947283 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 76/159] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:05.948637 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d940f-83fb-4abf-8a91-9f05a0b5c092 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:05.948924 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d940f-83fb-4abf-8a91-9f05a0b5c092 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:05.949204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d940f-83fb-4abf-8a91-9f05a0b5c092 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:05.950604 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 77/160] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:05.956557 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3cdeb-405c-4420-9124-592057ff9d53 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:05.956841 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3cdeb-405c-4420-9124-592057ff9d53 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:05.957116 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3cdeb-405c-4420-9124-592057ff9d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:05.977402 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-7fd3cdeb-405c-4420-9124-592057ff9d53 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:05.978599 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-7fd3cdeb-405c-4420-9124-592057ff9d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:05.998723 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 84/161] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 15:12:06.012082 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab33523-306f-4462-b52e-110a567cb1e4 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:06.052329 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 78/162] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:06.052978 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd3cdeb-405c-4420-9124-592057ff9d53 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:06.143941 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:06.171282 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd3cdeb-405c-4420-9124-592057ff9d53 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:06.204425 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab33523-306f-4462-b52e-110a567cb1e4 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:06.260812 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd6f61-dbf0-4ebf-89d8-73b571a18bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:06.262068 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd6f61-dbf0-4ebf-89d8-73b571a18bfe None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:06.262550 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd6f61-dbf0-4ebf-89d8-73b571a18bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:06.263022 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd6f61-dbf0-4ebf-89d8-73b571a18bfe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:06.265751 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da27644-8785-49b1-a2f4-87279559daf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:06.266506 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da27644-8785-49b1-a2f4-87279559daf0 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:06.266799 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da27644-8785-49b1-a2f4-87279559daf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:06.267076 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da27644-8785-49b1-a2f4-87279559daf0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:06.290289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd6f61-dbf0-4ebf-89d8-73b571a18bfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:06.295213 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-12dd6f61-dbf0-4ebf-89d8-73b571a18bfe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:12:06.295213 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:06.295213 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:06.295213 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:06.295213 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:06.295213 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:06.295213 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:06.295213 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:06.295213 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:06.295213 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:06.295213 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:06.295213 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:06.295213 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:06.295213 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:06.295213 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:06.295213 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:06.296190 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:06.296190 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:06.296190 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:06.296190 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:06.296190 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:06.296190 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:06.296190 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:06.296190 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:06.296190 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:06.296190 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:06.296190 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:06.296190 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:06.296190 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:06.296190 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:06.296190 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:06.296190 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:06.296981 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:06.296981 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:06.296981 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:06.296981 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:06.296981 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:06.296981 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:06.296981 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:06.296981 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:06.296981 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:06.296981 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:06.296981 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:12:06.296981 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:06.296981 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 85/163] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 15:12:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:12:06.297603 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd6f61-dbf0-4ebf-89d8-73b571a18bfe None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:06.305598 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da27644-8785-49b1-a2f4-87279559daf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:06.311017 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-8da27644-8785-49b1-a2f4-87279559daf0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 15:12:06.311017 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:06.311017 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:06.311017 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:06.311017 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:06.311017 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:06.311017 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:06.311017 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:06.311017 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:06.311017 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:06.311017 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:06.311017 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:06.311017 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:06.311017 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:06.311017 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:06.311017 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:06.311855 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:06.311855 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:06.311855 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:06.311855 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:06.311855 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:06.311855 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:06.311855 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:06.311855 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:06.311855 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:06.311855 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:06.311855 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:06.311855 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:06.311855 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:06.311855 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:06.311855 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:06.311855 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:06.312683 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:06.312683 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:06.312683 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:06.312683 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:06.312683 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:06.312683 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:06.312683 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:06.312683 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:06.312683 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:06.312683 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:06.312683 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 15:12:06.312683 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:06.320476 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 79/164] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:12:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:06.321199 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da27644-8785-49b1-a2f4-87279559daf0 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:06.360091 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:06.484100 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd6f61-dbf0-4ebf-89d8-73b571a18bfe None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:06.537728 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da27644-8785-49b1-a2f4-87279559daf0 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:06.554218 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12459ee3-849b-4638-9b3b-4a6b7678b075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:06.555452 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12459ee3-849b-4638-9b3b-4a6b7678b075 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:06.555790 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12459ee3-849b-4638-9b3b-4a6b7678b075 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:06.556006 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12459ee3-849b-4638-9b3b-4a6b7678b075 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:06.579492 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12459ee3-849b-4638-9b3b-4a6b7678b075 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:06.585199 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-12459ee3-849b-4638-9b3b-4a6b7678b075 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 15:12:06.585199 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:06.585199 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:06.585199 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:06.585199 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:06.585199 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:06.585199 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:06.585199 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:06.585199 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:06.585199 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:06.585199 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:06.585199 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:06.585199 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:06.585199 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:06.585199 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:06.585199 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:06.586313 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:06.586313 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:06.586313 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:06.586313 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:06.586313 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:06.586313 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:06.586313 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:06.586313 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:06.586313 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:06.586313 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:06.586313 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:06.586313 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:06.586313 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:06.586313 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:06.586313 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:06.586313 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:06.587086 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:06.587086 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:06.587086 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:06.587086 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:06.587086 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:06.587086 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:06.587086 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:06.587086 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:06.587086 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:06.587086 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:06.587086 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 15:12:06.587086 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:06.587086 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 86/165] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 15:12:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:06.587729 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12459ee3-849b-4638-9b3b-4a6b7678b075 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:06.625169 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a5c78c-46cf-4d3e-be7f-689a034ed24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:06.625950 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5c78c-46cf-4d3e-be7f-689a034ed24c None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:06.626239 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5c78c-46cf-4d3e-be7f-689a034ed24c None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:06.626515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5c78c-46cf-4d3e-be7f-689a034ed24c None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:06.627258 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a5c78c-46cf-4d3e-be7f-689a034ed24c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:06.627567 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a5c78c-46cf-4d3e-be7f-689a034ed24c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:06.629160 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a5c78c-46cf-4d3e-be7f-689a034ed24c None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:06.647845 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 80/166] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:12:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:06.648635 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a5c78c-46cf-4d3e-be7f-689a034ed24c None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:06.674161 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:06.744951 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:06.758019 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12459ee3-849b-4638-9b3b-4a6b7678b075 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:06.805071 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f01e6-4cdc-46fa-b28d-57d72d27d427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:06.805880 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f01e6-4cdc-46fa-b28d-57d72d27d427 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:06.806117 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f01e6-4cdc-46fa-b28d-57d72d27d427 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:06.806399 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f01e6-4cdc-46fa-b28d-57d72d27d427 None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:06.807111 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f01e6-4cdc-46fa-b28d-57d72d27d427 None admin] RBAC: Adding query filter params (name=member) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:06.807418 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f01e6-4cdc-46fa-b28d-57d72d27d427 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:06.809680 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f01e6-4cdc-46fa-b28d-57d72d27d427 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:06.818417 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 87/167] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 15:12:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:06.819234 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f01e6-4cdc-46fa-b28d-57d72d27d427 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:06.873067 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a5c78c-46cf-4d3e-be7f-689a034ed24c None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:06.960053 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d99b875-fd96-46f6-92f0-26fe3e42a47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:06.960718 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99b875-fd96-46f6-92f0-26fe3e42a47e None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:06.961007 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99b875-fd96-46f6-92f0-26fe3e42a47e None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:06.961290 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99b875-fd96-46f6-92f0-26fe3e42a47e None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:06.962016 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d99b875-fd96-46f6-92f0-26fe3e42a47e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:06.962328 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d99b875-fd96-46f6-92f0-26fe3e42a47e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:06.963261 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f01e6-4cdc-46fa-b28d-57d72d27d427 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:06.963768 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d99b875-fd96-46f6-92f0-26fe3e42a47e None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:06.987857 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 81/168] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:12:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:06.988744 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d99b875-fd96-46f6-92f0-26fe3e42a47e None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:07.034850 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67724099-37f9-435b-a23b-7a2302627930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:07.035511 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67724099-37f9-435b-a23b-7a2302627930 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:07.035796 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67724099-37f9-435b-a23b-7a2302627930 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:07.036093 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67724099-37f9-435b-a23b-7a2302627930 None admin] PATH_INFO: `/v3/users/338271833df8486b9ef50964b5ec7797` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:07.037323 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67724099-37f9-435b-a23b-7a2302627930 None admin] RBAC: Authorizing `identity:get_user(user_id=338271833df8486b9ef50964b5ec7797)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:07.039466 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67724099-37f9-435b-a23b-7a2302627930 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:07.041889 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 88/169] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:12:06 2022] GET /identity/v3/users/338271833df8486b9ef50964b5ec7797 => generated 247 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:07.042678 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67724099-37f9-435b-a23b-7a2302627930 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:07.070264 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:07.163889 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:07.229428 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67724099-37f9-435b-a23b-7a2302627930 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:07.245426 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d99b875-fd96-46f6-92f0-26fe3e42a47e None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:07.317409 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535c25e4-7870-49f2-9902-d5ca3faa670b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:07.318072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c25e4-7870-49f2-9902-d5ca3faa670b None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:07.318356 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c25e4-7870-49f2-9902-d5ca3faa670b None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:07.318637 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c25e4-7870-49f2-9902-d5ca3faa670b None admin] PATH_INFO: `/v3/users/a6cf4078f01748e98571f285e560294e` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:07.324993 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535c25e4-7870-49f2-9902-d5ca3faa670b None admin] RBAC: Authorizing `identity:get_user(user_id=a6cf4078f01748e98571f285e560294e)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:07.326452 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535c25e4-7870-49f2-9902-d5ca3faa670b None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:07.328269 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47944a4-5ec6-45ba-b406-51acbaf0fee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:07.328499 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 89/170] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:12:07 2022] GET /identity/v3/users/a6cf4078f01748e98571f285e560294e => generated 275 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:07.329119 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47944a4-5ec6-45ba-b406-51acbaf0fee0 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:07.329294 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535c25e4-7870-49f2-9902-d5ca3faa670b None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:07.329400 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47944a4-5ec6-45ba-b406-51acbaf0fee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:07.329783 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47944a4-5ec6-45ba-b406-51acbaf0fee0 None admin] PATH_INFO: `/v3/users/a6cf4078f01748e98571f285e560294e` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:07.331074 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47944a4-5ec6-45ba-b406-51acbaf0fee0 None admin] RBAC: Authorizing `identity:get_user(user_id=a6cf4078f01748e98571f285e560294e)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:07.332617 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47944a4-5ec6-45ba-b406-51acbaf0fee0 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:07.334743 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 82/171] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:12:06 2022] GET /identity/v3/users/a6cf4078f01748e98571f285e560294e => generated 275 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:07.338296 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47944a4-5ec6-45ba-b406-51acbaf0fee0 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:07.480583 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:07.484769 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535c25e4-7870-49f2-9902-d5ca3faa670b None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:07.528540 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd25045-4393-49ff-8b1b-4b32b912bedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:07.529160 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd25045-4393-49ff-8b1b-4b32b912bedf None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:07.529448 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd25045-4393-49ff-8b1b-4b32b912bedf None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:07.529723 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd25045-4393-49ff-8b1b-4b32b912bedf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:07.560178 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47944a4-5ec6-45ba-b406-51acbaf0fee0 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:07.597037 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4efa3-1b48-4769-b46f-b011c9a58f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:07.597578 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4efa3-1b48-4769-b46f-b011c9a58f37 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:07.597858 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4efa3-1b48-4769-b46f-b011c9a58f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:07.598199 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4efa3-1b48-4769-b46f-b011c9a58f37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:07.610055 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:07.657575 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd25045-4393-49ff-8b1b-4b32b912bedf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:07.659089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd25045-4393-49ff-8b1b-4b32b912bedf None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:07.672340 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e4efa3-1b48-4769-b46f-b011c9a58f37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:07.673989 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e4efa3-1b48-4769-b46f-b011c9a58f37 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:07.782124 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 83/172] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:12:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 15:12:07.787541 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4efa3-1b48-4769-b46f-b011c9a58f37 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:07.810231 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:07.828851 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 90/173] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:12:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 15:12:07.830814 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4c0a9-1f2a-4848-8738-828b43dcde89 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:07.831106 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4c0a9-1f2a-4848-8738-828b43dcde89 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:07.831389 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4c0a9-1f2a-4848-8738-828b43dcde89 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:07.832719 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 91/174] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:07.834087 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:07.835487 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5969932-4d2c-4879-a304-5580c6de3865 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:07.835779 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5969932-4d2c-4879-a304-5580c6de3865 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:07.836059 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5969932-4d2c-4879-a304-5580c6de3865 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:07.838146 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 92/175] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:07.838861 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5969932-4d2c-4879-a304-5580c6de3865 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:07.906849 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4efa3-1b48-4769-b46f-b011c9a58f37 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:07.945310 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4017e212-f520-41df-9752-b238196626ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:07.946322 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017e212-f520-41df-9752-b238196626ee None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:07.946846 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017e212-f520-41df-9752-b238196626ee None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:07.947313 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017e212-f520-41df-9752-b238196626ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:07.953447 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5969932-4d2c-4879-a304-5580c6de3865 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:07.990482 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791af2f-25f1-4d7d-a1a4-27490a8b91a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:07.991695 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791af2f-25f1-4d7d-a1a4-27490a8b91a3 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:07.991988 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791af2f-25f1-4d7d-a1a4-27490a8b91a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:07.992299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791af2f-25f1-4d7d-a1a4-27490a8b91a3 None admin] PATH_INFO: `/v3/projects/b972ca1ff8ee4c558eeb45ec24303061` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:07.993589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9791af2f-25f1-4d7d-a1a4-27490a8b91a3 None admin] RBAC: Authorizing `identity:get_project(project_id=b972ca1ff8ee4c558eeb45ec24303061)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:07.995170 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9791af2f-25f1-4d7d-a1a4-27490a8b91a3 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:08.001696 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 93/176] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:12:07 2022] GET /identity/v3/projects/b972ca1ff8ee4c558eeb45ec24303061 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:08.001696 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791af2f-25f1-4d7d-a1a4-27490a8b91a3 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:08.030898 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4017e212-f520-41df-9752-b238196626ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:08.032475 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4017e212-f520-41df-9752-b238196626ee None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:08.061866 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:08.077355 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:08.109519 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791af2f-25f1-4d7d-a1a4-27490a8b91a3 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:08.137484 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 84/177] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:12:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 15:12:08.139101 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908a458-ad6e-4e77-9aae-4a20991f2056 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:08.139393 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908a458-ad6e-4e77-9aae-4a20991f2056 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:08.139676 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908a458-ad6e-4e77-9aae-4a20991f2056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:08.153006 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d1224d-2fdc-4fb0-aa6d-da694e8b8437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:08.153668 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1224d-2fdc-4fb0-aa6d-da694e8b8437 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:08.153950 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1224d-2fdc-4fb0-aa6d-da694e8b8437 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:08.154230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1224d-2fdc-4fb0-aa6d-da694e8b8437 None admin] PATH_INFO: `/v3/projects/b972ca1ff8ee4c558eeb45ec24303061` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:08.155397 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d1224d-2fdc-4fb0-aa6d-da694e8b8437 None admin] RBAC: Authorizing `identity:get_project(project_id=b972ca1ff8ee4c558eeb45ec24303061)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:08.156980 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d1224d-2fdc-4fb0-aa6d-da694e8b8437 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:08.157198 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-3908a458-ad6e-4e77-9aae-4a20991f2056 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:08.158405 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-3908a458-ad6e-4e77-9aae-4a20991f2056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:08.158620 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 94/178] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:12:07 2022] GET /identity/v3/projects/b972ca1ff8ee4c558eeb45ec24303061 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:08.159733 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1cfe67-78e4-4c8d-9b81-5d40e6cd9de3 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:08.160022 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1cfe67-78e4-4c8d-9b81-5d40e6cd9de3 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:08.160326 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1cfe67-78e4-4c8d-9b81-5d40e6cd9de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:08.182430 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-3d1cfe67-78e4-4c8d-9b81-5d40e6cd9de3 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:08.183718 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-3d1cfe67-78e4-4c8d-9b81-5d40e6cd9de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:08.235938 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 85/179] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:08.236859 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3908a458-ad6e-4e77-9aae-4a20991f2056 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:08.265517 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 95/180] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:08.276507 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38887392-1663-4d1f-b8c5-7e4b3bc1714d None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:08.276802 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38887392-1663-4d1f-b8c5-7e4b3bc1714d None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:08.277083 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38887392-1663-4d1f-b8c5-7e4b3bc1714d None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:08.278394 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 96/181] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:08.279092 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38887392-1663-4d1f-b8c5-7e4b3bc1714d None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:08.340214 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3908a458-ad6e-4e77-9aae-4a20991f2056 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:08.353892 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:08.358406 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:08.379543 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dfe78d-d3a8-452e-b27d-e827f00573de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:08.380204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfe78d-d3a8-452e-b27d-e827f00573de None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:08.380492 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfe78d-d3a8-452e-b27d-e827f00573de None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:08.380779 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfe78d-d3a8-452e-b27d-e827f00573de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:08.381585 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dfe78d-d3a8-452e-b27d-e827f00573de None admin] RBAC: Adding query filter params (role.id=944427ca14ee4c81a05ec67e54f50e7f, scope.project.id=b972ca1ff8ee4c558eeb45ec24303061, user.id=a6cf4078f01748e98571f285e560294e) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:08.381933 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dfe78d-d3a8-452e-b27d-e827f00573de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:08.383462 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dfe78d-d3a8-452e-b27d-e827f00573de None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:08.389697 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 86/182] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:12:08 2022] GET /identity/v3/role_assignments?user.id=a6cf4078f01748e98571f285e560294e&scope.project.id=b972ca1ff8ee4c558eeb45ec24303061&role.id=944427ca14ee4c81a05ec67e54f50e7f => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:08.390310 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dfe78d-d3a8-452e-b27d-e827f00573de None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:08.401373 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38887392-1663-4d1f-b8c5-7e4b3bc1714d None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:08.441743 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d53dd5-9645-47ac-9659-069f4871e083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:08.442446 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d53dd5-9645-47ac-9659-069f4871e083 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:08.442738 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d53dd5-9645-47ac-9659-069f4871e083 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:08.443021 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d53dd5-9645-47ac-9659-069f4871e083 None admin] PATH_INFO: `/v3/projects/eab4ef6110f1406cbd92b046713d575b` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:08.444501 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d53dd5-9645-47ac-9659-069f4871e083 None admin] RBAC: Authorizing `identity:get_project(project_id=eab4ef6110f1406cbd92b046713d575b)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:08.446008 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d53dd5-9645-47ac-9659-069f4871e083 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:08.452988 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 97/183] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:12:08 2022] GET /identity/v3/projects/eab4ef6110f1406cbd92b046713d575b => generated 312 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:08.452988 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d53dd5-9645-47ac-9659-069f4871e083 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:08.505883 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dfe78d-d3a8-452e-b27d-e827f00573de None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:08.544636 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0193bd-9137-44df-abc5-0e72b3c0fc63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:08.545234 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0193bd-9137-44df-abc5-0e72b3c0fc63 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:08.545518 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0193bd-9137-44df-abc5-0e72b3c0fc63 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:08.545796 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0193bd-9137-44df-abc5-0e72b3c0fc63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:08.546594 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0193bd-9137-44df-abc5-0e72b3c0fc63 None admin] RBAC: Adding query filter params (role.id=7c88c815db854954b516ab97cd471647, scope.project.id=b972ca1ff8ee4c558eeb45ec24303061, user.id=338271833df8486b9ef50964b5ec7797) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:08.546979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0193bd-9137-44df-abc5-0e72b3c0fc63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:08.548488 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0193bd-9137-44df-abc5-0e72b3c0fc63 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:08.551830 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:08.558020 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 87/184] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:12:08 2022] GET /identity/v3/role_assignments?user.id=338271833df8486b9ef50964b5ec7797&scope.project.id=b972ca1ff8ee4c558eeb45ec24303061&role.id=7c88c815db854954b516ab97cd471647 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:08.559313 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d6e454-e319-409a-b415-43ae4b35a0f5 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:08.559688 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d6e454-e319-409a-b415-43ae4b35a0f5 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:08.559946 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d6e454-e319-409a-b415-43ae4b35a0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:08.583142 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-32d6e454-e319-409a-b415-43ae4b35a0f5 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:08.583522 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d53dd5-9645-47ac-9659-069f4871e083 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:08.585981 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-32d6e454-e319-409a-b415-43ae4b35a0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:08.618522 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:08.641103 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d8b459-2aa3-48e7-806e-e2ee19008cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:08.642333 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8b459-2aa3-48e7-806e-e2ee19008cbb None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:08.643393 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8b459-2aa3-48e7-806e-e2ee19008cbb None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:08.643884 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8b459-2aa3-48e7-806e-e2ee19008cbb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:08.665190 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d8b459-2aa3-48e7-806e-e2ee19008cbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:08.670543 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-b7d8b459-2aa3-48e7-806e-e2ee19008cbb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:12:08.670543 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:08.670543 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:08.670543 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:08.670543 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:08.670543 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:08.670543 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:08.670543 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:08.670543 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:08.670543 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:08.670543 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:08.670543 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:08.670543 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:08.670543 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:08.670543 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:08.670543 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:08.671719 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:08.671719 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:08.671719 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:08.671719 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:08.671719 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:08.671719 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:08.671719 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:08.671719 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:08.671719 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:08.671719 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:08.671719 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:08.671719 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:08.671719 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:08.671719 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:08.671719 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:08.671719 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:08.672785 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:08.672785 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:08.672785 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:08.672785 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:08.672785 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:08.672785 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:08.672785 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:08.672785 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:08.672785 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:08.672785 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:08.672785 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:12:08.672785 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:08.674909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 98/185] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 15:12:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:08.675924 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d8b459-2aa3-48e7-806e-e2ee19008cbb None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:08.688223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 88/186] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:08.689772 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75446b-5d6a-4160-80cd-3060ccb07a90 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:08.690066 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75446b-5d6a-4160-80cd-3060ccb07a90 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:08.690343 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75446b-5d6a-4160-80cd-3060ccb07a90 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:08.691956 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 89/187] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:08.692536 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca75446b-5d6a-4160-80cd-3060ccb07a90 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:08.794701 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d8b459-2aa3-48e7-806e-e2ee19008cbb None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:08.820768 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca75446b-5d6a-4160-80cd-3060ccb07a90 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:08.829334 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52d27a-7a79-4b22-a5a1-ae05db90978d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:08.829978 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52d27a-7a79-4b22-a5a1-ae05db90978d None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:08.830263 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52d27a-7a79-4b22-a5a1-ae05db90978d None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:08.830543 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52d27a-7a79-4b22-a5a1-ae05db90978d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:08.841119 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a52d27a-7a79-4b22-a5a1-ae05db90978d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:08.846027 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-6a52d27a-7a79-4b22-a5a1-ae05db90978d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 15:12:08.846027 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:08.846027 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:08.846027 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:08.846027 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:08.846027 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:08.846027 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:08.846027 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:08.846027 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:08.846027 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:08.846027 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:08.846027 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:08.846027 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:08.846027 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:08.846027 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:08.846027 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:08.847022 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:08.847022 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:08.847022 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:08.847022 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:08.847022 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:08.847022 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:08.847022 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:08.847022 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:08.847022 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:08.847022 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:08.847022 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:08.847022 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:08.847022 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:08.847022 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:08.847022 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:08.847022 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:08.847831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:08.847831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:08.847831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:08.847831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:08.847831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:08.847831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:08.847831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:08.847831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:08.847831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:08.847831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:08.847831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 15:12:08.847831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:08.847831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 99/188] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 15:12:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:08.849026 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52d27a-7a79-4b22-a5a1-ae05db90978d None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:08.849967 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:08.855423 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd99461-a56b-447f-9dba-22a98ca6b62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:08.856040 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd99461-a56b-447f-9dba-22a98ca6b62d None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:08.856367 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd99461-a56b-447f-9dba-22a98ca6b62d None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:08.856648 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd99461-a56b-447f-9dba-22a98ca6b62d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:08.857445 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd99461-a56b-447f-9dba-22a98ca6b62d None admin] RBAC: Adding query filter params (role.id=d6c91838e4944cfbb23e1b6ecb009d26, scope.project.id=eab4ef6110f1406cbd92b046713d575b, user.id=a6cf4078f01748e98571f285e560294e) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:08.857765 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd99461-a56b-447f-9dba-22a98ca6b62d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:08.859274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd99461-a56b-447f-9dba-22a98ca6b62d None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:08.865113 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 90/189] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:12:08 2022] GET /identity/v3/role_assignments?user.id=a6cf4078f01748e98571f285e560294e&scope.project.id=eab4ef6110f1406cbd92b046713d575b&role.id=d6c91838e4944cfbb23e1b6ecb009d26 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:08.866272 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab952b83-49c7-412a-a32c-47298963e95a None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:08.881458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab952b83-49c7-412a-a32c-47298963e95a None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:08.881458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab952b83-49c7-412a-a32c-47298963e95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:08.889457 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-ab952b83-49c7-412a-a32c-47298963e95a None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:08.890749 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-ab952b83-49c7-412a-a32c-47298963e95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:08.895087 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:08.961958 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52d27a-7a79-4b22-a5a1-ae05db90978d None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:08.970374 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 91/190] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:08.973245 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab952b83-49c7-412a-a32c-47298963e95a None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:09.007968 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df57574f-ea62-4f5a-9b25-f96da8535bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:09.008640 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57574f-ea62-4f5a-9b25-f96da8535bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:09.008928 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57574f-ea62-4f5a-9b25-f96da8535bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:09.009199 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57574f-ea62-4f5a-9b25-f96da8535bf7 None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:09.009932 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df57574f-ea62-4f5a-9b25-f96da8535bf7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:09.010494 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df57574f-ea62-4f5a-9b25-f96da8535bf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:09.011664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df57574f-ea62-4f5a-9b25-f96da8535bf7 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:09.026042 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 100/191] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:12:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:09.026042 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df57574f-ea62-4f5a-9b25-f96da8535bf7 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:09.094463 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab952b83-49c7-412a-a32c-47298963e95a None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:09.121167 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:09.133212 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a333f6e8-a3c0-4f93-b7da-48398b0c1277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:09.133858 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333f6e8-a3c0-4f93-b7da-48398b0c1277 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:09.137717 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333f6e8-a3c0-4f93-b7da-48398b0c1277 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:09.137717 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333f6e8-a3c0-4f93-b7da-48398b0c1277 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:09.137717 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a333f6e8-a3c0-4f93-b7da-48398b0c1277 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:09.137717 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a333f6e8-a3c0-4f93-b7da-48398b0c1277 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:09.138020 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df57574f-ea62-4f5a-9b25-f96da8535bf7 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:09.138613 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 92/192] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:12:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:09.139377 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a333f6e8-a3c0-4f93-b7da-48398b0c1277 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:09.144425 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:09.178371 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0536fb9-69bd-4892-ac61-9f61acbef803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:09.178991 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0536fb9-69bd-4892-ac61-9f61acbef803 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:09.179281 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0536fb9-69bd-4892-ac61-9f61acbef803 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:09.179558 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0536fb9-69bd-4892-ac61-9f61acbef803 None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:09.180298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0536fb9-69bd-4892-ac61-9f61acbef803 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:09.180608 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0536fb9-69bd-4892-ac61-9f61acbef803 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:09.181965 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0536fb9-69bd-4892-ac61-9f61acbef803 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:09.189771 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 101/193] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:12:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:09.190552 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0536fb9-69bd-4892-ac61-9f61acbef803 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:09.246655 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a333f6e8-a3c0-4f93-b7da-48398b0c1277 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:09.280122 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db785a9-7e33-4088-a3d4-0b995b33857d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:09.280760 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db785a9-7e33-4088-a3d4-0b995b33857d None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:09.281045 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db785a9-7e33-4088-a3d4-0b995b33857d None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:09.281324 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db785a9-7e33-4088-a3d4-0b995b33857d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:09.290829 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0536fb9-69bd-4892-ac61-9f61acbef803 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:09.291558 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db785a9-7e33-4088-a3d4-0b995b33857d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:09.297597 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-1db785a9-7e33-4088-a3d4-0b995b33857d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 15:12:09.297597 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:09.297597 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:09.297597 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:09.297597 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:09.297597 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:09.297597 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:09.297597 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:09.297597 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:09.297597 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:09.297597 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:09.297597 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:09.297597 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:09.297597 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:09.297597 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:09.297597 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:09.298655 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:09.298655 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:09.298655 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:09.298655 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:09.298655 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:09.298655 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:09.298655 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:09.298655 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:09.298655 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:09.298655 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:09.298655 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:09.298655 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:09.298655 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:09.298655 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:09.298655 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:09.298655 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:09.299635 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:09.299635 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:09.299635 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:09.299635 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:09.299635 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:09.299635 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:09.299635 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:09.299635 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:09.299635 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:09.299635 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:09.299635 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 15:12:09.299635 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:09.299635 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 93/194] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:12:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:09.304766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db785a9-7e33-4088-a3d4-0b995b33857d None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:09.327349 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085e60e0-eaa4-4e5f-8b05-0fa38fce0838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:09.327986 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e60e0-eaa4-4e5f-8b05-0fa38fce0838 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:09.328303 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e60e0-eaa4-4e5f-8b05-0fa38fce0838 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:09.328582 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e60e0-eaa4-4e5f-8b05-0fa38fce0838 None admin] PATH_INFO: `/v3/users/f9487fe9b610445596a324d7de416e52` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:09.341387 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085e60e0-eaa4-4e5f-8b05-0fa38fce0838 None admin] RBAC: Authorizing `identity:get_user(user_id=f9487fe9b610445596a324d7de416e52)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:09.342827 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085e60e0-eaa4-4e5f-8b05-0fa38fce0838 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:09.344489 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 102/195] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:12:09 2022] GET /identity/v3/users/f9487fe9b610445596a324d7de416e52 => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:09.345249 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085e60e0-eaa4-4e5f-8b05-0fa38fce0838 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:09.354645 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:09.373629 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:09.411821 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db785a9-7e33-4088-a3d4-0b995b33857d None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:09.447763 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac675d-7dcf-4ae3-9955-0470e8daa6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:09.448405 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac675d-7dcf-4ae3-9955-0470e8daa6b0 None admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:09.448693 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac675d-7dcf-4ae3-9955-0470e8daa6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:09.448969 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac675d-7dcf-4ae3-9955-0470e8daa6b0 None admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:09.449643 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ac675d-7dcf-4ae3-9955-0470e8daa6b0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:09.450422 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085e60e0-eaa4-4e5f-8b05-0fa38fce0838 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:09.451098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ac675d-7dcf-4ae3-9955-0470e8daa6b0 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:09.483044 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3ad6d0-af9d-4617-ba8e-223ad4a417dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:09.483672 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ad6d0-af9d-4617-ba8e-223ad4a417dc None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:09.483953 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ad6d0-af9d-4617-ba8e-223ad4a417dc None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:09.484254 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ad6d0-af9d-4617-ba8e-223ad4a417dc None admin] PATH_INFO: `/v3/users/f9487fe9b610445596a324d7de416e52` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:09.485422 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3ad6d0-af9d-4617-ba8e-223ad4a417dc None admin] RBAC: Authorizing `identity:get_user(user_id=f9487fe9b610445596a324d7de416e52)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:09.486835 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3ad6d0-af9d-4617-ba8e-223ad4a417dc None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:09.488453 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 103/196] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:12:09 2022] GET /identity/v3/users/f9487fe9b610445596a324d7de416e52 => generated 283 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:09.489203 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3ad6d0-af9d-4617-ba8e-223ad4a417dc None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:09.514941 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 94/197] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 5 15:12:09 2022] POST /identity/v3/users => generated 297 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:12:09.521167 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac675d-7dcf-4ae3-9955-0470e8daa6b0 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:09.590569 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:09.600799 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3ad6d0-af9d-4617-ba8e-223ad4a417dc None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:09.641217 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6597d67-3e0b-4a2a-850a-4b2bdec43527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:09.641888 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6597d67-3e0b-4a2a-850a-4b2bdec43527 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:09.642178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6597d67-3e0b-4a2a-850a-4b2bdec43527 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:09.642459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6597d67-3e0b-4a2a-850a-4b2bdec43527 None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:09.642692 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac675d-7dcf-4ae3-9955-0470e8daa6b0 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:09.643271 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6597d67-3e0b-4a2a-850a-4b2bdec43527 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:09.643594 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6597d67-3e0b-4a2a-850a-4b2bdec43527 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:09.645051 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6597d67-3e0b-4a2a-850a-4b2bdec43527 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:09.651070 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:09.660698 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 104/198] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 15:12:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:09.660698 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6597d67-3e0b-4a2a-850a-4b2bdec43527 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:09.690150 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a20352-aff9-415e-8218-ecd0dbda9dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:09.691698 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a20352-aff9-415e-8218-ecd0dbda9dbe None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:09.691992 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a20352-aff9-415e-8218-ecd0dbda9dbe None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:09.692304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a20352-aff9-415e-8218-ecd0dbda9dbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:09.779466 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6597d67-3e0b-4a2a-850a-4b2bdec43527 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:09.785872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a20352-aff9-415e-8218-ecd0dbda9dbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:09.787376 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a20352-aff9-415e-8218-ecd0dbda9dbe None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:09.821222 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba32299-c112-4330-827b-9b1fd05ef0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:09.821862 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba32299-c112-4330-827b-9b1fd05ef0be None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:09.822149 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba32299-c112-4330-827b-9b1fd05ef0be None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:09.822434 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba32299-c112-4330-827b-9b1fd05ef0be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:09.852332 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:09.909795 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba32299-c112-4330-827b-9b1fd05ef0be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:09.911288 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba32299-c112-4330-827b-9b1fd05ef0be None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:09.926630 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 95/199] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:12:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 15:12:09.928225 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a20352-aff9-415e-8218-ecd0dbda9dbe None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:09.939345 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:10.017880 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 105/200] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:12:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 15:12:10.018627 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba32299-c112-4330-827b-9b1fd05ef0be None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:10.070857 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a20352-aff9-415e-8218-ecd0dbda9dbe None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:10.078440 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:10.123756 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d3746-8a3f-4fad-949c-367fe95c959b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:10.125352 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d3746-8a3f-4fad-949c-367fe95c959b None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:10.125643 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d3746-8a3f-4fad-949c-367fe95c959b None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:10.125925 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d3746-8a3f-4fad-949c-367fe95c959b None admin] PATH_INFO: `/v3/users/5d9236f58acd40bcbaf379964e48e4a2` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:10.127212 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d3746-8a3f-4fad-949c-367fe95c959b None admin] RBAC: Authorizing `identity:get_user(user_id=5d9236f58acd40bcbaf379964e48e4a2)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:10.127969 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba32299-c112-4330-827b-9b1fd05ef0be None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:10.128785 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d3746-8a3f-4fad-949c-367fe95c959b None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:10.130906 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 96/201] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:12:09 2022] GET /identity/v3/users/5d9236f58acd40bcbaf379964e48e4a2 => generated 289 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:10.132527 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d3746-8a3f-4fad-949c-367fe95c959b None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:10.180526 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6787b1-dee0-4fe8-9320-766a59888645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:10.181495 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6787b1-dee0-4fe8-9320-766a59888645 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:10.182239 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6787b1-dee0-4fe8-9320-766a59888645 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:10.183613 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6787b1-dee0-4fe8-9320-766a59888645 None admin] PATH_INFO: `/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:10.185091 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6787b1-dee0-4fe8-9320-766a59888645 None admin] RBAC: Authorizing `identity:get_project(project_id=759a514c6fa54b4b8fa90f2d4f181f99)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:10.186746 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6787b1-dee0-4fe8-9320-766a59888645 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:10.189275 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 106/202] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:12:10 2022] GET /identity/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:10.190394 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6787b1-dee0-4fe8-9320-766a59888645 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:10.233114 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:10.259492 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d3746-8a3f-4fad-949c-367fe95c959b None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:10.297585 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4559924-8586-4309-a2a6-8697b49f63d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:10.298626 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4559924-8586-4309-a2a6-8697b49f63d1 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:10.299111 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4559924-8586-4309-a2a6-8697b49f63d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:10.299584 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4559924-8586-4309-a2a6-8697b49f63d1 None admin] PATH_INFO: `/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:10.301023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4559924-8586-4309-a2a6-8697b49f63d1 None admin] RBAC: Authorizing `identity:get_project(project_id=759a514c6fa54b4b8fa90f2d4f181f99)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:10.302732 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4559924-8586-4309-a2a6-8697b49f63d1 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:10.304703 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 97/203] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:12:10 2022] GET /identity/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:10.305791 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4559924-8586-4309-a2a6-8697b49f63d1 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:10.307663 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6787b1-dee0-4fe8-9320-766a59888645 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:10.331662 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:10.344610 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea5af90-c8d6-4682-8105-e82c22065636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:10.345784 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5af90-c8d6-4682-8105-e82c22065636 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:10.345784 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5af90-c8d6-4682-8105-e82c22065636 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:10.345963 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5af90-c8d6-4682-8105-e82c22065636 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:10.415843 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4559924-8586-4309-a2a6-8697b49f63d1 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:10.419417 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea5af90-c8d6-4682-8105-e82c22065636 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:10.420933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea5af90-c8d6-4682-8105-e82c22065636 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:10.453226 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa21a39-b526-4f3a-b2d4-fe79f85f9be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:10.454181 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa21a39-b526-4f3a-b2d4-fe79f85f9be0 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:10.454669 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa21a39-b526-4f3a-b2d4-fe79f85f9be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:10.455152 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa21a39-b526-4f3a-b2d4-fe79f85f9be0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:10.456180 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa21a39-b526-4f3a-b2d4-fe79f85f9be0 None admin] RBAC: Adding query filter params (role.id=7c88c815db854954b516ab97cd471647, scope.project.id=759a514c6fa54b4b8fa90f2d4f181f99, user.id=f9487fe9b610445596a324d7de416e52) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:10.456695 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa21a39-b526-4f3a-b2d4-fe79f85f9be0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:10.458371 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa21a39-b526-4f3a-b2d4-fe79f85f9be0 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:10.465029 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 98/204] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:12:10 2022] GET /identity/v3/role_assignments?user.id=f9487fe9b610445596a324d7de416e52&scope.project.id=759a514c6fa54b4b8fa90f2d4f181f99&role.id=7c88c815db854954b516ab97cd471647 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:10.466204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa21a39-b526-4f3a-b2d4-fe79f85f9be0 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:10.475658 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:10.529569 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 107/205] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:12:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 15:12:10.535326 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea5af90-c8d6-4682-8105-e82c22065636 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:10.569600 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:10.582674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa21a39-b526-4f3a-b2d4-fe79f85f9be0 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:10.622383 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2b24d-fc74-45f1-b885-86cb038e568f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:10.623413 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2b24d-fc74-45f1-b885-86cb038e568f None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:10.623893 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2b24d-fc74-45f1-b885-86cb038e568f None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:10.624451 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2b24d-fc74-45f1-b885-86cb038e568f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:10.625771 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b2b24d-fc74-45f1-b885-86cb038e568f None admin] RBAC: Adding query filter params (role.id=944427ca14ee4c81a05ec67e54f50e7f, scope.project.id=759a514c6fa54b4b8fa90f2d4f181f99, user.id=f9487fe9b610445596a324d7de416e52) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:10.626310 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b2b24d-fc74-45f1-b885-86cb038e568f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:10.628011 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b2b24d-fc74-45f1-b885-86cb038e568f None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:10.635316 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 99/206] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:12:10 2022] GET /identity/v3/role_assignments?user.id=f9487fe9b610445596a324d7de416e52&scope.project.id=759a514c6fa54b4b8fa90f2d4f181f99&role.id=944427ca14ee4c81a05ec67e54f50e7f => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:10.654969 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea5af90-c8d6-4682-8105-e82c22065636 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:10.692958 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06268876-cab2-48fd-adad-b59c69ed60f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:10.693628 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06268876-cab2-48fd-adad-b59c69ed60f2 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:10.693911 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06268876-cab2-48fd-adad-b59c69ed60f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:10.694723 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06268876-cab2-48fd-adad-b59c69ed60f2 None admin] PATH_INFO: `/v3/projects/b972ca1ff8ee4c558eeb45ec24303061` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:10.695973 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06268876-cab2-48fd-adad-b59c69ed60f2 None admin] RBAC: Authorizing `identity:get_project(project_id=b972ca1ff8ee4c558eeb45ec24303061)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:10.697492 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06268876-cab2-48fd-adad-b59c69ed60f2 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:10.699447 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 108/207] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:12:10 2022] GET /identity/v3/projects/b972ca1ff8ee4c558eeb45ec24303061 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:10.705208 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2b24d-fc74-45f1-b885-86cb038e568f None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:10.820182 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:10.854188 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2b24d-fc74-45f1-b885-86cb038e568f None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:10.891693 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e8c22-6e1e-4034-8a4b-576f67780e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:10.892347 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e8c22-6e1e-4034-8a4b-576f67780e80 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:10.892630 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e8c22-6e1e-4034-8a4b-576f67780e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:10.892906 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e8c22-6e1e-4034-8a4b-576f67780e80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:10.893702 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8e8c22-6e1e-4034-8a4b-576f67780e80 None admin] RBAC: Adding query filter params (role.id=09be81c3953242a08b57fd702a5082e3, scope.project.id=b972ca1ff8ee4c558eeb45ec24303061, user.id=5d9236f58acd40bcbaf379964e48e4a2) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:10.894017 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8e8c22-6e1e-4034-8a4b-576f67780e80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:10.895467 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8e8c22-6e1e-4034-8a4b-576f67780e80 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:10.901661 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 100/208] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:12:10 2022] GET /identity/v3/role_assignments?user.id=5d9236f58acd40bcbaf379964e48e4a2&scope.project.id=b972ca1ff8ee4c558eeb45ec24303061&role.id=09be81c3953242a08b57fd702a5082e3 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:11.781810 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468378a-1b5c-4a50-8061-9df641b9b976 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:11.782321 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468378a-1b5c-4a50-8061-9df641b9b976 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:11.782427 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468378a-1b5c-4a50-8061-9df641b9b976 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:11.783731 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 109/209] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:11.789754 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175ed43-0f06-4e90-bfda-dbaab8564c7b None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:11.790180 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175ed43-0f06-4e90-bfda-dbaab8564c7b None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:11.790353 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175ed43-0f06-4e90-bfda-dbaab8564c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:11.808847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-b175ed43-0f06-4e90-bfda-dbaab8564c7b None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:11.810056 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-b175ed43-0f06-4e90-bfda-dbaab8564c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:11.880318 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 101/210] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 15:12:11.904474 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e468378a-1b5c-4a50-8061-9df641b9b976 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:11.942991 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8260742c-5c70-419a-832f-d992ff4c800a None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:11.943637 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8260742c-5c70-419a-832f-d992ff4c800a None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:11.944690 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8260742c-5c70-419a-832f-d992ff4c800a None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:11.946273 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 102/211] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 15:12:11.951564 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f4bbf-a93f-413a-ab6a-4aee6e46a3f9 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:11.952125 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f4bbf-a93f-413a-ab6a-4aee6e46a3f9 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:11.952750 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f4bbf-a93f-413a-ab6a-4aee6e46a3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:11.972699 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-994f4bbf-a93f-413a-ab6a-4aee6e46a3f9 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:11.973892 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-994f4bbf-a93f-413a-ab6a-4aee6e46a3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:12.020421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:12.025653 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e468378a-1b5c-4a50-8061-9df641b9b976 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:12.049908 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:12.063529 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 103/212] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 15:12:12.076103 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d3e88c-36ed-4c53-8e4b-00ca6232a4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:12.077084 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3e88c-36ed-4c53-8e4b-00ca6232a4de None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:12.077575 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3e88c-36ed-4c53-8e4b-00ca6232a4de None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:12.078078 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3e88c-36ed-4c53-8e4b-00ca6232a4de None admin] PATH_INFO: `/v3/roles/member` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:12.089191 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f4bbf-a93f-413a-ab6a-4aee6e46a3f9 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:12.089641 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d3e88c-36ed-4c53-8e4b-00ca6232a4de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:12.095922 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-58d3e88c-36ed-4c53-8e4b-00ca6232a4de None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 15:12:12.095922 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:12.095922 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:12.095922 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:12.095922 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:12.095922 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:12.095922 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:12.095922 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:12.095922 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:12.095922 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:12.095922 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:12.095922 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:12.095922 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:12.095922 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:12.095922 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:12.095922 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:12.096822 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:12.096822 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:12.096822 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:12.096822 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:12.096822 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:12.096822 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:12.096822 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:12.096822 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:12.096822 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:12.096822 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:12.096822 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:12.096822 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:12.096822 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:12.096822 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:12.096822 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:12.096822 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:12.097597 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:12.097597 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:12.097597 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:12.097597 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:12.097597 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:12.097597 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:12.097597 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:12.097597 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:12.097597 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:12.097597 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:12.097597 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 15:12:12.097597 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:12.097597 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 110/213] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:12:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:12.105056 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d3e88c-36ed-4c53-8e4b-00ca6232a4de None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:12.215236 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f4bbf-a93f-413a-ab6a-4aee6e46a3f9 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:12.235928 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d3e88c-36ed-4c53-8e4b-00ca6232a4de None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:12.252585 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b879d6c-3be1-45fb-b537-e64982cb12d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:12.253593 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b879d6c-3be1-45fb-b537-e64982cb12d5 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:12.254032 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b879d6c-3be1-45fb-b537-e64982cb12d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:12.254548 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b879d6c-3be1-45fb-b537-e64982cb12d5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:12.255908 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b879d6c-3be1-45fb-b537-e64982cb12d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:12.258420 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b879d6c-3be1-45fb-b537-e64982cb12d5 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:12.260324 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 104/214] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:12:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:12:12.266214 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b879d6c-3be1-45fb-b537-e64982cb12d5 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:12.278981 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebafa4d7-990d-4032-9188-17d6fb1a053c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:12.279879 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebafa4d7-990d-4032-9188-17d6fb1a053c None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:12.280392 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebafa4d7-990d-4032-9188-17d6fb1a053c None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:12.280895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebafa4d7-990d-4032-9188-17d6fb1a053c None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:12.281822 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebafa4d7-990d-4032-9188-17d6fb1a053c None admin] RBAC: Adding query filter params (name=member) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:12.282338 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebafa4d7-990d-4032-9188-17d6fb1a053c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:12.283907 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebafa4d7-990d-4032-9188-17d6fb1a053c None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:12.292735 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 111/215] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 15:12:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:12.298966 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebafa4d7-990d-4032-9188-17d6fb1a053c None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:12.317464 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:12.368534 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:12.393977 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b879d6c-3be1-45fb-b537-e64982cb12d5 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:12.426821 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebafa4d7-990d-4032-9188-17d6fb1a053c None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:12.439179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99052e6b-7027-4e50-b28e-036eef456cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:12.440002 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99052e6b-7027-4e50-b28e-036eef456cc5 None admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:12.440488 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99052e6b-7027-4e50-b28e-036eef456cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:12.441258 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99052e6b-7027-4e50-b28e-036eef456cc5 None admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:12.442397 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99052e6b-7027-4e50-b28e-036eef456cc5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:12.444772 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99052e6b-7027-4e50-b28e-036eef456cc5 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:12.469073 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177aa090-40f1-4e86-b376-bddff5470a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:12.472294 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177aa090-40f1-4e86-b376-bddff5470a0e None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:12.472294 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177aa090-40f1-4e86-b376-bddff5470a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:12.472294 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177aa090-40f1-4e86-b376-bddff5470a0e None admin] PATH_INFO: `/v3/users/f87a2184ca15477f95e0693ad00ae819` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:12.485281 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177aa090-40f1-4e86-b376-bddff5470a0e None admin] RBAC: Authorizing `identity:get_user(user_id=f87a2184ca15477f95e0693ad00ae819)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:12.486738 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177aa090-40f1-4e86-b376-bddff5470a0e None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:12.488686 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 112/216] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:12:12 2022] GET /identity/v3/users/f87a2184ca15477f95e0693ad00ae819 => generated 297 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:12.493915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177aa090-40f1-4e86-b376-bddff5470a0e None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:12.563365 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 105/217] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 5 15:12:12 2022] POST /identity/v3/users => generated 297 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:12:12.573395 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:12.612227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177aa090-40f1-4e86-b376-bddff5470a0e None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:12.655554 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb7b6d-93a3-44d9-8157-04dccaf576d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:12.656241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb7b6d-93a3-44d9-8157-04dccaf576d9 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:12.656528 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb7b6d-93a3-44d9-8157-04dccaf576d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:12.656809 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb7b6d-93a3-44d9-8157-04dccaf576d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:12.737482 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb7b6d-93a3-44d9-8157-04dccaf576d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:12.739089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb7b6d-93a3-44d9-8157-04dccaf576d9 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:12.812780 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:12.848447 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 113/218] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:12:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 15:12:12.852591 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99052e6b-7027-4e50-b28e-036eef456cc5 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:12.855546 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8884661-b7d3-4d29-96d9-5e68f50f47e6 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:12.855887 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8884661-b7d3-4d29-96d9-5e68f50f47e6 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:12.856338 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8884661-b7d3-4d29-96d9-5e68f50f47e6 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:12.857735 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 114/219] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 15:12:12.859027 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ee2ee-bf79-40ff-9177-c4b4a1c9ed4f None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:12.859352 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ee2ee-bf79-40ff-9177-c4b4a1c9ed4f None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:12.859667 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ee2ee-bf79-40ff-9177-c4b4a1c9ed4f None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:12.860924 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 115/220] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:12.863045 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742da504-91d7-4d20-ab47-f40be35c6d3c None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:12.863374 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742da504-91d7-4d20-ab47-f40be35c6d3c None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:12.863688 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742da504-91d7-4d20-ab47-f40be35c6d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:12.884925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-742da504-91d7-4d20-ab47-f40be35c6d3c None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:12.886515 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-742da504-91d7-4d20-ab47-f40be35c6d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:12.970420 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 116/221] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 15:12:12.972273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dccfc2-43d5-474e-8a38-a4be9d447ad8 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:12.972569 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dccfc2-43d5-474e-8a38-a4be9d447ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:12.972844 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dccfc2-43d5-474e-8a38-a4be9d447ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:12.992457 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-b0dccfc2-43d5-474e-8a38-a4be9d447ad8 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:12.994320 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-b0dccfc2-43d5-474e-8a38-a4be9d447ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:12.995770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99052e6b-7027-4e50-b28e-036eef456cc5 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:13.025194 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:13.046478 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c136a66-04f0-40ee-82a2-feabb7ca10b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:13.047139 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c136a66-04f0-40ee-82a2-feabb7ca10b0 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:13.047428 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c136a66-04f0-40ee-82a2-feabb7ca10b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:13.048327 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c136a66-04f0-40ee-82a2-feabb7ca10b0 None admin] PATH_INFO: `/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:13.049553 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c136a66-04f0-40ee-82a2-feabb7ca10b0 None admin] RBAC: Authorizing `identity:get_project(project_id=759a514c6fa54b4b8fa90f2d4f181f99)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:13.051559 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c136a66-04f0-40ee-82a2-feabb7ca10b0 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:13.054305 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 106/222] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:12:12 2022] GET /identity/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:13.056152 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c136a66-04f0-40ee-82a2-feabb7ca10b0 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:13.085052 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 117/223] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:13.086357 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dccfc2-43d5-474e-8a38-a4be9d447ad8 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:13.130050 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:13.179457 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c136a66-04f0-40ee-82a2-feabb7ca10b0 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:13.215770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a212e4d-e378-411a-98b1-cd70fb55fc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:13.216788 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a212e4d-e378-411a-98b1-cd70fb55fc2f None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:13.217309 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a212e4d-e378-411a-98b1-cd70fb55fc2f None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:13.217722 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a212e4d-e378-411a-98b1-cd70fb55fc2f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:13.223870 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dccfc2-43d5-474e-8a38-a4be9d447ad8 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:13.233236 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a212e4d-e378-411a-98b1-cd70fb55fc2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:13.242274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-8a212e4d-e378-411a-98b1-cd70fb55fc2f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 15:12:13.242274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:13.242274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:13.242274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:13.242274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:13.242274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:13.242274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:13.242274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:13.242274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:13.242274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:13.242274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:13.242274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:13.242274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:13.242274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:13.242274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:13.242274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:13.243154 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:13.243154 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:13.243154 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:13.243154 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:13.243154 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:13.243154 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:13.243154 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:13.243154 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:13.243154 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:13.243154 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:13.243154 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:13.243154 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:13.243154 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:13.243154 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:13.243154 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:13.243154 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:13.243925 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:13.243925 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:13.243925 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:13.243925 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:13.243925 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:13.243925 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:13.243925 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:13.243925 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:13.243925 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:13.243925 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:13.243925 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 15:12:13.243925 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:13.246433 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 107/224] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 15:12:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:13.247419 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a212e4d-e378-411a-98b1-cd70fb55fc2f None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:13.265741 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faac9fd-a540-46d3-af79-3d2b868477c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:13.266751 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faac9fd-a540-46d3-af79-3d2b868477c9 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:13.267045 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faac9fd-a540-46d3-af79-3d2b868477c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:13.267329 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faac9fd-a540-46d3-af79-3d2b868477c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:13.268441 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faac9fd-a540-46d3-af79-3d2b868477c9 None admin] RBAC: Adding query filter params (role.id=d6c91838e4944cfbb23e1b6ecb009d26, scope.project.id=759a514c6fa54b4b8fa90f2d4f181f99, user.id=f87a2184ca15477f95e0693ad00ae819) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:13.268763 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faac9fd-a540-46d3-af79-3d2b868477c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:13.270366 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faac9fd-a540-46d3-af79-3d2b868477c9 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:13.277286 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 118/225] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:12:13 2022] GET /identity/v3/role_assignments?user.id=f87a2184ca15477f95e0693ad00ae819&scope.project.id=759a514c6fa54b4b8fa90f2d4f181f99&role.id=d6c91838e4944cfbb23e1b6ecb009d26 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:13.278962 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faac9fd-a540-46d3-af79-3d2b868477c9 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:13.331166 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:13.442981 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:13.446153 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a212e4d-e378-411a-98b1-cd70fb55fc2f None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:13.450834 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faac9fd-a540-46d3-af79-3d2b868477c9 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:13.510134 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d8068a-0bbe-41fa-9166-8eb825d9cc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:13.511538 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d8068a-0bbe-41fa-9166-8eb825d9cc2f None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:13.512353 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d8068a-0bbe-41fa-9166-8eb825d9cc2f None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:13.513043 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d8068a-0bbe-41fa-9166-8eb825d9cc2f None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:13.514652 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d8068a-0bbe-41fa-9166-8eb825d9cc2f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:13.515398 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d8068a-0bbe-41fa-9166-8eb825d9cc2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:13.520423 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d8068a-0bbe-41fa-9166-8eb825d9cc2f None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:13.524990 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a25c3b7-6a9d-47f7-aaab-31ed2ec7cb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:13.527936 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25c3b7-6a9d-47f7-aaab-31ed2ec7cb96 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:13.528784 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25c3b7-6a9d-47f7-aaab-31ed2ec7cb96 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:13.530779 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25c3b7-6a9d-47f7-aaab-31ed2ec7cb96 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:13.550465 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 119/226] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:12:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:13.559794 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d8068a-0bbe-41fa-9166-8eb825d9cc2f None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:13.573076 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a25c3b7-6a9d-47f7-aaab-31ed2ec7cb96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:13.578656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-0a25c3b7-6a9d-47f7-aaab-31ed2ec7cb96 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:12:13.578656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:13.578656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:13.578656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:13.578656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:13.578656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:13.578656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:13.578656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:13.578656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:13.578656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:13.578656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:13.578656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:13.578656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:13.578656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:13.578656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:13.578656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:13.580266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:13.580266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:13.580266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:13.580266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:13.580266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:13.580266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:13.580266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:13.580266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:13.580266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:13.580266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:13.580266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:13.580266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:13.580266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:13.580266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:13.580266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:13.580266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:13.581749 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:13.581749 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:13.581749 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:13.581749 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:13.581749 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:13.581749 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:13.581749 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:13.581749 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:13.581749 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:13.581749 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:13.581749 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:12:13.581749 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:13.589246 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 108/227] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 15:12:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:13.609081 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a25c3b7-6a9d-47f7-aaab-31ed2ec7cb96 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:13.763634 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d8068a-0bbe-41fa-9166-8eb825d9cc2f None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:13.823618 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a853f-7e55-49ed-9538-eec7764d3d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:13.825941 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a853f-7e55-49ed-9538-eec7764d3d32 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:13.826754 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a853f-7e55-49ed-9538-eec7764d3d32 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:13.827504 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a853f-7e55-49ed-9538-eec7764d3d32 None admin] PATH_INFO: `/v3/users/f9487fe9b610445596a324d7de416e52` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:13.829794 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a853f-7e55-49ed-9538-eec7764d3d32 None admin] RBAC: Authorizing `identity:get_user(user_id=f9487fe9b610445596a324d7de416e52)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:13.832169 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a25c3b7-6a9d-47f7-aaab-31ed2ec7cb96 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:13.833068 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a853f-7e55-49ed-9538-eec7764d3d32 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:13.838384 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 120/228] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:12:13 2022] GET /identity/v3/users/f9487fe9b610445596a324d7de416e52 => generated 283 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:12:13.840417 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:13.843984 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a853f-7e55-49ed-9538-eec7764d3d32 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:13.871267 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:13.889772 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aaca8d-088c-4e88-88da-d7c7b24a712d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:13.890431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aaca8d-088c-4e88-88da-d7c7b24a712d None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:13.890694 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aaca8d-088c-4e88-88da-d7c7b24a712d None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:13.890985 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aaca8d-088c-4e88-88da-d7c7b24a712d None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:13.892001 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aaca8d-088c-4e88-88da-d7c7b24a712d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:13.892001 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aaca8d-088c-4e88-88da-d7c7b24a712d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:13.893454 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aaca8d-088c-4e88-88da-d7c7b24a712d None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:13.913758 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 109/229] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:12:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:13.921942 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aaca8d-088c-4e88-88da-d7c7b24a712d None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:13.977429 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a853f-7e55-49ed-9538-eec7764d3d32 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:14.013219 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd12a1ca-501a-4be4-bde1-d1b5f5e1c471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:14.014308 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd12a1ca-501a-4be4-bde1-d1b5f5e1c471 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:14.014808 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd12a1ca-501a-4be4-bde1-d1b5f5e1c471 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:14.015281 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd12a1ca-501a-4be4-bde1-d1b5f5e1c471 None admin] PATH_INFO: `/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:14.016881 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd12a1ca-501a-4be4-bde1-d1b5f5e1c471 None admin] RBAC: Authorizing `identity:get_project(project_id=759a514c6fa54b4b8fa90f2d4f181f99)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:14.018605 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd12a1ca-501a-4be4-bde1-d1b5f5e1c471 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:14.020759 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 121/230] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:12:13 2022] GET /identity/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:12:14.027387 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd12a1ca-501a-4be4-bde1-d1b5f5e1c471 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:14.143744 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aaca8d-088c-4e88-88da-d7c7b24a712d None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:14.220491 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:14.230770 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd12a1ca-501a-4be4-bde1-d1b5f5e1c471 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:14.245356 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c62d04-8ce7-4752-8b51-4a50a8c0ca81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:14.246029 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c62d04-8ce7-4752-8b51-4a50a8c0ca81 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:14.246322 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c62d04-8ce7-4752-8b51-4a50a8c0ca81 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:14.246604 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c62d04-8ce7-4752-8b51-4a50a8c0ca81 None admin] PATH_INFO: `/v3/users/f9487fe9b610445596a324d7de416e52` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:14.253058 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c62d04-8ce7-4752-8b51-4a50a8c0ca81 None admin] RBAC: Authorizing `identity:get_user(user_id=f9487fe9b610445596a324d7de416e52)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:14.254543 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c62d04-8ce7-4752-8b51-4a50a8c0ca81 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:14.256754 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 110/231] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:12:13 2022] GET /identity/v3/users/f9487fe9b610445596a324d7de416e52 => generated 283 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:12:14.262209 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c62d04-8ce7-4752-8b51-4a50a8c0ca81 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:14.267418 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78b769-7722-438c-bfba-6fc0ecd061bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:14.268035 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78b769-7722-438c-bfba-6fc0ecd061bc None admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:14.268353 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78b769-7722-438c-bfba-6fc0ecd061bc None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:14.268633 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78b769-7722-438c-bfba-6fc0ecd061bc None admin] PATH_INFO: `/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99/users/f9487fe9b610445596a324d7de416e52/roles/944427ca14ee4c81a05ec67e54f50e7f` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:14.271382 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da78b769-7722-438c-bfba-6fc0ecd061bc None admin] RBAC: Authorizing `identity:create_grant(project_id=759a514c6fa54b4b8fa90f2d4f181f99, user_id=f9487fe9b610445596a324d7de416e52, role_id=944427ca14ee4c81a05ec67e54f50e7f)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:14.273432 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da78b769-7722-438c-bfba-6fc0ecd061bc None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:14.312215 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:14.359855 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 122/232] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 15:12:14 2022] PUT /identity/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99/users/f9487fe9b610445596a324d7de416e52/roles/944427ca14ee4c81a05ec67e54f50e7f => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:12:14.396434 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c62d04-8ce7-4752-8b51-4a50a8c0ca81 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:14.436453 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e6f261-ade4-4aa7-88c7-352f321ce8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:14.437112 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6f261-ade4-4aa7-88c7-352f321ce8f9 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:14.437401 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6f261-ade4-4aa7-88c7-352f321ce8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:14.437735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6f261-ade4-4aa7-88c7-352f321ce8f9 None admin] PATH_INFO: `/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:14.438991 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e6f261-ade4-4aa7-88c7-352f321ce8f9 None admin] RBAC: Authorizing `identity:get_project(project_id=759a514c6fa54b4b8fa90f2d4f181f99)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:14.440504 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e6f261-ade4-4aa7-88c7-352f321ce8f9 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:14.442188 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 111/233] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:12:14 2022] GET /identity/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:12:14.449006 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78b769-7722-438c-bfba-6fc0ecd061bc None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:14.581171 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78b769-7722-438c-bfba-6fc0ecd061bc None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:14.620976 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a1fd75-88bc-42a9-afcc-b7e0e0119077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:14.621689 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a1fd75-88bc-42a9-afcc-b7e0e0119077 None admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:14.621978 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a1fd75-88bc-42a9-afcc-b7e0e0119077 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:14.622262 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a1fd75-88bc-42a9-afcc-b7e0e0119077 None admin] PATH_INFO: `/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99/users/f9487fe9b610445596a324d7de416e52/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:14.624343 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a1fd75-88bc-42a9-afcc-b7e0e0119077 None admin] RBAC: Authorizing `identity:create_grant(project_id=759a514c6fa54b4b8fa90f2d4f181f99, user_id=f9487fe9b610445596a324d7de416e52, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:14.626067 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a1fd75-88bc-42a9-afcc-b7e0e0119077 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:14.719089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 123/234] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 15:12:14 2022] PUT /identity/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99/users/f9487fe9b610445596a324d7de416e52/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:12:15.106965 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f201d-f7c1-4765-9f49-4a674520de71 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:15.107485 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f201d-f7c1-4765-9f49-4a674520de71 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:15.107485 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f201d-f7c1-4765-9f49-4a674520de71 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:15.109209 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 112/235] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:15.115106 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ade892-be3b-4477-aace-72e747b3d107 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:15.115624 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ade892-be3b-4477-aace-72e747b3d107 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:15.115821 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ade892-be3b-4477-aace-72e747b3d107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:15.140176 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-a8ade892-be3b-4477-aace-72e747b3d107 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:15.141382 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-a8ade892-be3b-4477-aace-72e747b3d107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:15.150175 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:15.241494 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b37d39-ee48-4a69-90c1-16bb75ef20e4 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:15.241783 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b37d39-ee48-4a69-90c1-16bb75ef20e4 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:15.242226 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b37d39-ee48-4a69-90c1-16bb75ef20e4 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:15.243667 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 113/236] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:15.248638 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3c554-e1ee-491e-bd1d-d481dc8bc7fd None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:15.249009 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3c554-e1ee-491e-bd1d-d481dc8bc7fd None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:15.249114 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 124/237] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:15.249376 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3c554-e1ee-491e-bd1d-d481dc8bc7fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:15.268920 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-01c3c554-e1ee-491e-bd1d-d481dc8bc7fd None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:15.270226 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-01c3c554-e1ee-491e-bd1d-d481dc8bc7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:15.293935 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ade892-be3b-4477-aace-72e747b3d107 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:15.345303 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 114/238] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:15.370896 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c3c554-e1ee-491e-bd1d-d481dc8bc7fd None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:15.416646 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:15.498217 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c3c554-e1ee-491e-bd1d-d481dc8bc7fd None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:15.502217 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ade892-be3b-4477-aace-72e747b3d107 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:15.562436 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb8ff4-537b-49c6-807f-e896aec7070c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:15.563087 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb8ff4-537b-49c6-807f-e896aec7070c None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:15.563375 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb8ff4-537b-49c6-807f-e896aec7070c None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:15.563654 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb8ff4-537b-49c6-807f-e896aec7070c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:15.565243 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eb8ff4-537b-49c6-807f-e896aec7070c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:15.566698 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eb8ff4-537b-49c6-807f-e896aec7070c None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:15.568480 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 115/239] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:12:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:12:15.574375 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb8ff4-537b-49c6-807f-e896aec7070c None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:15.577842 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a6066-81b8-4992-bbfa-06c52b8a1bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:15.578649 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a6066-81b8-4992-bbfa-06c52b8a1bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:15.578939 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a6066-81b8-4992-bbfa-06c52b8a1bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:15.579215 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a6066-81b8-4992-bbfa-06c52b8a1bb9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:15.604161 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32a6066-81b8-4992-bbfa-06c52b8a1bb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:15.609416 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-b32a6066-81b8-4992-bbfa-06c52b8a1bb9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 15:12:15.609416 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:15.609416 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:15.609416 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:15.609416 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:15.609416 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:15.609416 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:15.609416 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:15.609416 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:15.609416 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:15.609416 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:15.609416 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:15.609416 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:15.609416 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:15.609416 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:15.609416 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:15.610288 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:15.610288 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:15.610288 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:15.610288 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:15.610288 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:15.610288 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:15.610288 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:15.610288 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:15.610288 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:15.610288 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:15.610288 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:15.610288 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:15.610288 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:15.610288 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:15.610288 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:15.610288 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:15.611085 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:15.611085 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:15.611085 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:15.611085 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:15.611085 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:15.611085 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:15.611085 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:15.611085 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:15.611085 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:15.611085 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:15.611085 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 15:12:15.611085 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:15.611085 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 125/240] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:12:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:12:15.619764 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a6066-81b8-4992-bbfa-06c52b8a1bb9 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:15.625359 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:15.695298 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb8ff4-537b-49c6-807f-e896aec7070c None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:15.697280 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:15.759915 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b590b2-4c3e-4558-a7f3-ba8e4305e426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:15.760577 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b590b2-4c3e-4558-a7f3-ba8e4305e426 None admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:15.760864 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b590b2-4c3e-4558-a7f3-ba8e4305e426 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:15.761146 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b590b2-4c3e-4558-a7f3-ba8e4305e426 None admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:15.761851 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b590b2-4c3e-4558-a7f3-ba8e4305e426 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:15.763324 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b590b2-4c3e-4558-a7f3-ba8e4305e426 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:15.805016 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a6066-81b8-4992-bbfa-06c52b8a1bb9 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:15.838710 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5fb5e4-a375-40a7-83a2-9b81fe780d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:15.839321 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5fb5e4-a375-40a7-83a2-9b81fe780d6c None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:15.839607 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5fb5e4-a375-40a7-83a2-9b81fe780d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:15.839883 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5fb5e4-a375-40a7-83a2-9b81fe780d6c None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:15.840630 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5fb5e4-a375-40a7-83a2-9b81fe780d6c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:15.840943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5fb5e4-a375-40a7-83a2-9b81fe780d6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:15.842316 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5fb5e4-a375-40a7-83a2-9b81fe780d6c None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:15.854475 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 126/241] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 15:12:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:12:15.858330 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa1a4e-cfa9-4a57-a750-72322bd428dc None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:15.858330 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa1a4e-cfa9-4a57-a750-72322bd428dc None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:15.858330 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa1a4e-cfa9-4a57-a750-72322bd428dc None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:15.858330 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 127/242] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:15.868511 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fa1a4e-cfa9-4a57-a750-72322bd428dc None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:15.868584 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 116/243] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 5 15:12:15 2022] POST /identity/v3/users => generated 293 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:12:15.869955 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54875e-b0bb-47b1-9ab5-57da770174c5 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:15.870253 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54875e-b0bb-47b1-9ab5-57da770174c5 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:15.870537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54875e-b0bb-47b1-9ab5-57da770174c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:15.907334 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-ba54875e-b0bb-47b1-9ab5-57da770174c5 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:15.908513 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-ba54875e-b0bb-47b1-9ab5-57da770174c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:15.943990 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:15.991545 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fa1a4e-cfa9-4a57-a750-72322bd428dc None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:16.068689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 117/244] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:16.081608 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c44c54-78b9-4c3e-b7f0-25ffda35e90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:16.083252 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c44c54-78b9-4c3e-b7f0-25ffda35e90a None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:16.083756 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c44c54-78b9-4c3e-b7f0-25ffda35e90a None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:16.084054 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c44c54-78b9-4c3e-b7f0-25ffda35e90a None admin] PATH_INFO: `/v3/users/edb7b556ac304a179ea214a93a59e188` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:16.102905 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba54875e-b0bb-47b1-9ab5-57da770174c5 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:16.106459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c44c54-78b9-4c3e-b7f0-25ffda35e90a None admin] RBAC: Authorizing `identity:get_user(user_id=edb7b556ac304a179ea214a93a59e188)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:16.108296 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c44c54-78b9-4c3e-b7f0-25ffda35e90a None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:16.110476 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 128/245] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:12:15 2022] GET /identity/v3/users/edb7b556ac304a179ea214a93a59e188 => generated 297 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:16.119029 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c44c54-78b9-4c3e-b7f0-25ffda35e90a None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:16.217272 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:16.231318 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba54875e-b0bb-47b1-9ab5-57da770174c5 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:16.232237 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c44c54-78b9-4c3e-b7f0-25ffda35e90a None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:16.258220 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:16.269624 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f11de-b669-4eb0-bd82-cf34fdcfcce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:16.273576 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f11de-b669-4eb0-bd82-cf34fdcfcce7 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:16.273576 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f11de-b669-4eb0-bd82-cf34fdcfcce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:16.273576 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f11de-b669-4eb0-bd82-cf34fdcfcce7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:16.273783 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709664f-fc9f-4a11-8275-80cb3c289acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:16.273783 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709664f-fc9f-4a11-8275-80cb3c289acf None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:16.273783 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709664f-fc9f-4a11-8275-80cb3c289acf None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:16.273783 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709664f-fc9f-4a11-8275-80cb3c289acf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:16.282753 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79f11de-b669-4eb0-bd82-cf34fdcfcce7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:16.288363 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-b79f11de-b669-4eb0-bd82-cf34fdcfcce7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 15:12:16.288363 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:16.288363 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:16.288363 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:16.288363 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:16.288363 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:16.288363 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:16.288363 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:16.288363 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:16.288363 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:16.288363 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:16.288363 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:16.288363 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:16.288363 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:16.288363 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:16.288363 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:16.289337 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:16.289337 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:16.289337 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:16.289337 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:16.289337 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:16.289337 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:16.289337 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:16.289337 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:16.289337 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:16.289337 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:16.289337 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:16.289337 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:16.289337 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:16.289337 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:16.289337 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:16.289337 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:16.290718 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:16.290718 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:16.290718 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:16.290718 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:16.290718 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:16.290718 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:16.290718 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:16.290718 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:16.290718 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:16.290718 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:16.290718 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 15:12:16.290718 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:16.290718 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 118/246] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:12:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:12:16.295390 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f11de-b669-4eb0-bd82-cf34fdcfcce7 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:16.346366 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7709664f-fc9f-4a11-8275-80cb3c289acf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:16.347909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7709664f-fc9f-4a11-8275-80cb3c289acf None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:16.406792 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f11de-b669-4eb0-bd82-cf34fdcfcce7 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:16.448813 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5270b33c-ff28-4d16-9bbe-e063fc2b7b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:16.449364 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270b33c-ff28-4d16-9bbe-e063fc2b7b61 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:16.449745 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270b33c-ff28-4d16-9bbe-e063fc2b7b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:16.450026 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270b33c-ff28-4d16-9bbe-e063fc2b7b61 None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:16.450854 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5270b33c-ff28-4d16-9bbe-e063fc2b7b61 None admin] RBAC: Adding query filter params (name=member) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:16.451164 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5270b33c-ff28-4d16-9bbe-e063fc2b7b61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:16.452706 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5270b33c-ff28-4d16-9bbe-e063fc2b7b61 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:16.456967 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 129/247] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:12:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 15:12:16.463129 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709664f-fc9f-4a11-8275-80cb3c289acf None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:16.463706 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 119/248] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 15:12:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:12:16.469173 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5270b33c-ff28-4d16-9bbe-e063fc2b7b61 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:16.484479 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:16.489288 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:16.573774 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709664f-fc9f-4a11-8275-80cb3c289acf None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:16.583020 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5270b33c-ff28-4d16-9bbe-e063fc2b7b61 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:16.613605 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1ccb2d-517c-4ff1-8c56-4df49c5e04b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:16.614258 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ccb2d-517c-4ff1-8c56-4df49c5e04b4 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:16.614541 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ccb2d-517c-4ff1-8c56-4df49c5e04b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:16.614821 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ccb2d-517c-4ff1-8c56-4df49c5e04b4 None admin] PATH_INFO: `/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:16.616032 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1ccb2d-517c-4ff1-8c56-4df49c5e04b4 None admin] RBAC: Authorizing `identity:get_project(project_id=759a514c6fa54b4b8fa90f2d4f181f99)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:16.618377 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1ccb2d-517c-4ff1-8c56-4df49c5e04b4 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:16.620380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 130/249] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:12:16 2022] GET /identity/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:12:16.626575 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1ccb2d-517c-4ff1-8c56-4df49c5e04b4 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:16.630381 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101269d8-dc5e-4288-b755-bde11aec187f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:16.631597 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101269d8-dc5e-4288-b755-bde11aec187f None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:16.631890 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101269d8-dc5e-4288-b755-bde11aec187f None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:16.632201 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101269d8-dc5e-4288-b755-bde11aec187f None admin] PATH_INFO: `/v3/users/f87a2184ca15477f95e0693ad00ae819` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:16.634371 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101269d8-dc5e-4288-b755-bde11aec187f None admin] RBAC: Authorizing `identity:get_user(user_id=f87a2184ca15477f95e0693ad00ae819)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:16.635839 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101269d8-dc5e-4288-b755-bde11aec187f None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:16.637512 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 120/250] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:12:16 2022] GET /identity/v3/users/f87a2184ca15477f95e0693ad00ae819 => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:12:16.643590 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101269d8-dc5e-4288-b755-bde11aec187f None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:16.729806 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:16.738475 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:16.739967 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1ccb2d-517c-4ff1-8c56-4df49c5e04b4 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:16.760632 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101269d8-dc5e-4288-b755-bde11aec187f None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:16.776831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d7af8-b1e2-40cb-9ed8-bbd5601705ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:16.777488 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d7af8-b1e2-40cb-9ed8-bbd5601705ee None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:16.777781 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d7af8-b1e2-40cb-9ed8-bbd5601705ee None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:16.778061 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d7af8-b1e2-40cb-9ed8-bbd5601705ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:16.778915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450d7af8-b1e2-40cb-9ed8-bbd5601705ee None admin] RBAC: Adding query filter params (role.id=09be81c3953242a08b57fd702a5082e3, scope.project.id=759a514c6fa54b4b8fa90f2d4f181f99, user.id=edb7b556ac304a179ea214a93a59e188) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:16.779237 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450d7af8-b1e2-40cb-9ed8-bbd5601705ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:16.780719 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450d7af8-b1e2-40cb-9ed8-bbd5601705ee None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:16.786853 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 131/251] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:12:16 2022] GET /identity/v3/role_assignments?user.id=edb7b556ac304a179ea214a93a59e188&scope.project.id=759a514c6fa54b4b8fa90f2d4f181f99&role.id=09be81c3953242a08b57fd702a5082e3 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:16.804208 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8152f2-4247-415c-a25e-815fdc0198c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:16.805238 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8152f2-4247-415c-a25e-815fdc0198c9 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:16.805700 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8152f2-4247-415c-a25e-815fdc0198c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:16.806187 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8152f2-4247-415c-a25e-815fdc0198c9 None admin] PATH_INFO: `/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:16.807713 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8152f2-4247-415c-a25e-815fdc0198c9 None admin] RBAC: Authorizing `identity:get_project(project_id=759a514c6fa54b4b8fa90f2d4f181f99)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:16.809508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8152f2-4247-415c-a25e-815fdc0198c9 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:16.816201 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 121/252] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:12:16 2022] GET /identity/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:12:16.818502 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d7af8-b1e2-40cb-9ed8-bbd5601705ee None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:16.929821 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d7af8-b1e2-40cb-9ed8-bbd5601705ee None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:16.967167 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dadd2ed-bb71-4e5f-850b-4f46b0a2bfa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:16.968154 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dadd2ed-bb71-4e5f-850b-4f46b0a2bfa7 None admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:16.968608 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dadd2ed-bb71-4e5f-850b-4f46b0a2bfa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:16.969069 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dadd2ed-bb71-4e5f-850b-4f46b0a2bfa7 None admin] PATH_INFO: `/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99/users/f87a2184ca15477f95e0693ad00ae819/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:16.971261 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dadd2ed-bb71-4e5f-850b-4f46b0a2bfa7 None admin] RBAC: Authorizing `identity:create_grant(project_id=759a514c6fa54b4b8fa90f2d4f181f99, user_id=f87a2184ca15477f95e0693ad00ae819, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:16.972980 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dadd2ed-bb71-4e5f-850b-4f46b0a2bfa7 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:17.052373 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 132/253] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 15:12:16 2022] PUT /identity/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99/users/f87a2184ca15477f95e0693ad00ae819/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:12:17.210517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebf767-920b-4917-b2d4-098a8192caf7 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:17.211161 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebf767-920b-4917-b2d4-098a8192caf7 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:17.211635 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebf767-920b-4917-b2d4-098a8192caf7 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:17.213291 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 122/254] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:17.218778 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aefccb-4a2e-4297-aae2-d92cdcd43c32 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:17.219677 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aefccb-4a2e-4297-aae2-d92cdcd43c32 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:17.220213 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aefccb-4a2e-4297-aae2-d92cdcd43c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:17.246135 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-13aefccb-4a2e-4297-aae2-d92cdcd43c32 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:17.246445 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-13aefccb-4a2e-4297-aae2-d92cdcd43c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:17.260509 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:17.335355 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 133/255] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:17.378662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebf767-920b-4917-b2d4-098a8192caf7 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:17.381659 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a6c08-8c59-4589-8335-703ea23aec68 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:17.381659 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a6c08-8c59-4589-8335-703ea23aec68 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:17.381659 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a6c08-8c59-4589-8335-703ea23aec68 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:17.382809 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 134/256] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:17.398492 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c4514-061e-4761-8658-43fb4da1b766 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:17.398792 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c4514-061e-4761-8658-43fb4da1b766 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:17.399055 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c4514-061e-4761-8658-43fb4da1b766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:17.432104 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-fb5c4514-061e-4761-8658-43fb4da1b766 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:17.432104 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-fb5c4514-061e-4761-8658-43fb4da1b766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:17.544953 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebf767-920b-4917-b2d4-098a8192caf7 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:17.563091 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 135/257] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 15:12:17.591721 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa99263c-5b84-406a-aeef-ee90dff6e14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:17.592331 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99263c-5b84-406a-aeef-ee90dff6e14a None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:17.592618 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99263c-5b84-406a-aeef-ee90dff6e14a None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:17.592897 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99263c-5b84-406a-aeef-ee90dff6e14a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:17.605326 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa99263c-5b84-406a-aeef-ee90dff6e14a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:17.607634 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5c4514-061e-4761-8658-43fb4da1b766 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:17.616793 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-aa99263c-5b84-406a-aeef-ee90dff6e14a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 15:12:17.616793 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:17.616793 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:17.616793 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:17.616793 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:17.616793 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:17.616793 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:17.616793 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:17.616793 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:17.616793 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:17.616793 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:17.616793 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:17.616793 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:17.616793 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:17.616793 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:17.616793 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:17.617710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:17.617710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:17.617710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:17.617710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:17.617710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:17.617710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:17.617710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:17.617710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:17.617710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:17.617710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:17.617710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:17.617710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:17.617710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:17.617710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:17.617710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:17.617710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:17.618500 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:17.618500 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:17.618500 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:17.618500 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:17.618500 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:17.618500 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:17.618500 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:17.618500 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:17.618500 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:17.618500 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:17.618500 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 15:12:17.618500 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:17.618500 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 123/258] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 15:12:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:17.626603 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa99263c-5b84-406a-aeef-ee90dff6e14a None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:17.636171 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:17.788089 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa99263c-5b84-406a-aeef-ee90dff6e14a None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:17.829339 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:17.852188 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2704c9c8-04c6-49a4-bfe3-c65d65fbea69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:17.852845 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2704c9c8-04c6-49a4-bfe3-c65d65fbea69 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:17.853093 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2704c9c8-04c6-49a4-bfe3-c65d65fbea69 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:17.853377 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2704c9c8-04c6-49a4-bfe3-c65d65fbea69 None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:17.854169 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2704c9c8-04c6-49a4-bfe3-c65d65fbea69 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:17.854587 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2704c9c8-04c6-49a4-bfe3-c65d65fbea69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:17.855876 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2704c9c8-04c6-49a4-bfe3-c65d65fbea69 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:17.858747 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5c4514-061e-4761-8658-43fb4da1b766 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:17.870283 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 124/259] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:12:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:12:17.876028 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2704c9c8-04c6-49a4-bfe3-c65d65fbea69 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:17.897288 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39bae8e-f968-4253-acb7-6ed5aa332299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:17.898436 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39bae8e-f968-4253-acb7-6ed5aa332299 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:17.898935 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39bae8e-f968-4253-acb7-6ed5aa332299 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:17.899407 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39bae8e-f968-4253-acb7-6ed5aa332299 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:17.919659 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39bae8e-f968-4253-acb7-6ed5aa332299 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:17.925241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-c39bae8e-f968-4253-acb7-6ed5aa332299 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:12:17.925241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:17.925241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:17.925241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:17.925241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:17.925241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:17.925241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:17.925241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:17.925241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:17.925241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:17.925241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:17.925241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:17.925241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:17.925241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:17.925241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:17.925241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:17.926105 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:17.926105 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:17.926105 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:17.926105 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:17.926105 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:17.926105 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:17.926105 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:17.926105 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:17.926105 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:17.926105 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:17.926105 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:17.926105 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:17.926105 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:17.926105 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:17.926105 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:17.926105 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:17.926954 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:17.926954 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:17.926954 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:17.926954 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:17.926954 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:17.926954 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:17.926954 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:17.926954 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:17.926954 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:17.926954 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:17.926954 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:12:17.926954 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:17.936982 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 136/260] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 15:12:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:12:17.944559 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39bae8e-f968-4253-acb7-6ed5aa332299 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:18.104208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39bae8e-f968-4253-acb7-6ed5aa332299 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:18.155644 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d461c2d-b44e-4974-ad52-841f67f3ae91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:18.156166 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d461c2d-b44e-4974-ad52-841f67f3ae91 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:18.156451 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d461c2d-b44e-4974-ad52-841f67f3ae91 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:18.156730 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d461c2d-b44e-4974-ad52-841f67f3ae91 None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:18.157501 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d461c2d-b44e-4974-ad52-841f67f3ae91 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:18.157747 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d461c2d-b44e-4974-ad52-841f67f3ae91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:18.158175 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:18.159574 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d461c2d-b44e-4974-ad52-841f67f3ae91 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:18.161174 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:18.178681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 137/261] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:12:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:12:18.184608 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d461c2d-b44e-4974-ad52-841f67f3ae91 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:18.263833 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2704c9c8-04c6-49a4-bfe3-c65d65fbea69 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:18.337737 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a3c17f-5809-448a-ac9f-e7f5f2d6fda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:18.338781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3c17f-5809-448a-ac9f-e7f5f2d6fda9 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:18.339268 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3c17f-5809-448a-ac9f-e7f5f2d6fda9 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:18.339775 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3c17f-5809-448a-ac9f-e7f5f2d6fda9 None admin] PATH_INFO: `/v3/users/f9487fe9b610445596a324d7de416e52` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:18.341234 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a3c17f-5809-448a-ac9f-e7f5f2d6fda9 None admin] RBAC: Authorizing `identity:get_user(user_id=f9487fe9b610445596a324d7de416e52)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:18.342896 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a3c17f-5809-448a-ac9f-e7f5f2d6fda9 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:18.344753 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 125/262] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:12:17 2022] GET /identity/v3/users/f9487fe9b610445596a324d7de416e52 => generated 283 bytes in 469 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:12:18.350326 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a3c17f-5809-448a-ac9f-e7f5f2d6fda9 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:18.447916 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d461c2d-b44e-4974-ad52-841f67f3ae91 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:18.536892 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3749937-9cb6-45a0-82ba-dcf0e56ddcba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:18.537959 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3749937-9cb6-45a0-82ba-dcf0e56ddcba None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:18.538448 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3749937-9cb6-45a0-82ba-dcf0e56ddcba None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:18.538926 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3749937-9cb6-45a0-82ba-dcf0e56ddcba None admin] PATH_INFO: `/v3/users/f9487fe9b610445596a324d7de416e52` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:18.544780 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3749937-9cb6-45a0-82ba-dcf0e56ddcba None admin] RBAC: Authorizing `identity:get_user(user_id=f9487fe9b610445596a324d7de416e52)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:18.546543 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3749937-9cb6-45a0-82ba-dcf0e56ddcba None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:18.548747 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 138/263] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:12:18 2022] GET /identity/v3/users/f9487fe9b610445596a324d7de416e52 => generated 283 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:12:18.554275 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3749937-9cb6-45a0-82ba-dcf0e56ddcba None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:18.579985 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a3c17f-5809-448a-ac9f-e7f5f2d6fda9 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:18.629382 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f9e6c1-c91e-41fc-9d4d-f418605dce70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:18.630007 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9e6c1-c91e-41fc-9d4d-f418605dce70 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:18.630289 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9e6c1-c91e-41fc-9d4d-f418605dce70 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:18.630562 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9e6c1-c91e-41fc-9d4d-f418605dce70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:18.641356 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:18.725720 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:18.737043 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f9e6c1-c91e-41fc-9d4d-f418605dce70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:18.741757 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f9e6c1-c91e-41fc-9d4d-f418605dce70 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:18.806561 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3749937-9cb6-45a0-82ba-dcf0e56ddcba None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:18.875711 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19015bd-9bcc-49a2-8220-d527c1072655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:18.876657 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19015bd-9bcc-49a2-8220-d527c1072655 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:18.877935 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19015bd-9bcc-49a2-8220-d527c1072655 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:18.878423 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19015bd-9bcc-49a2-8220-d527c1072655 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:18.937934 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 126/264] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:12:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 588 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 15:12:18.943452 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f9e6c1-c91e-41fc-9d4d-f418605dce70 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:19.001096 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:19.011634 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19015bd-9bcc-49a2-8220-d527c1072655 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:19.013157 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19015bd-9bcc-49a2-8220-d527c1072655 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:19.095483 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f9e6c1-c91e-41fc-9d4d-f418605dce70 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:19.137381 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:19.167384 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a149474-3920-4277-a786-20bfd1b85c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:19.168142 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a149474-3920-4277-a786-20bfd1b85c4d None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:19.168431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a149474-3920-4277-a786-20bfd1b85c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:19.168708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a149474-3920-4277-a786-20bfd1b85c4d None admin] PATH_INFO: `/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:19.172807 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a149474-3920-4277-a786-20bfd1b85c4d None admin] RBAC: Authorizing `identity:get_project(project_id=759a514c6fa54b4b8fa90f2d4f181f99)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:19.174312 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a149474-3920-4277-a786-20bfd1b85c4d None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:19.176353 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 127/265] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:12:18 2022] GET /identity/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99 => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:12:19.184359 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a149474-3920-4277-a786-20bfd1b85c4d None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:19.222418 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 139/266] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:12:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 668 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 15:12:19.223527 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97da7f44-8b57-4429-8d50-1f6da6d5a40c None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:19.223818 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97da7f44-8b57-4429-8d50-1f6da6d5a40c None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:19.224116 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97da7f44-8b57-4429-8d50-1f6da6d5a40c None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:19.225933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 140/267] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:19.234985 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bbffa7-37a5-471e-9516-5defecc3843d None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:19.235270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bbffa7-37a5-471e-9516-5defecc3843d None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:19.235556 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bbffa7-37a5-471e-9516-5defecc3843d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:19.274117 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-f9bbffa7-37a5-471e-9516-5defecc3843d None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:19.275667 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-f9bbffa7-37a5-471e-9516-5defecc3843d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:19.370475 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a149474-3920-4277-a786-20bfd1b85c4d None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:19.377325 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:19.423726 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba5bdc3-9e0a-4dee-b9e6-f258138c2b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:19.424389 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5bdc3-9e0a-4dee-b9e6-f258138c2b91 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:19.425612 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5bdc3-9e0a-4dee-b9e6-f258138c2b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:19.425902 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5bdc3-9e0a-4dee-b9e6-f258138c2b91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:19.426717 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba5bdc3-9e0a-4dee-b9e6-f258138c2b91 None admin] RBAC: Adding query filter params (role.id=944427ca14ee4c81a05ec67e54f50e7f, scope.project.id=759a514c6fa54b4b8fa90f2d4f181f99, user.id=f9487fe9b610445596a324d7de416e52) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:19.427034 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba5bdc3-9e0a-4dee-b9e6-f258138c2b91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:19.428583 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba5bdc3-9e0a-4dee-b9e6-f258138c2b91 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:19.432045 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 141/268] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:19.433034 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bbffa7-37a5-471e-9516-5defecc3843d None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:19.446380 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 128/269] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:12:19 2022] GET /identity/v3/role_assignments?user.id=f9487fe9b610445596a324d7de416e52&scope.project.id=759a514c6fa54b4b8fa90f2d4f181f99&role.id=944427ca14ee4c81a05ec67e54f50e7f => generated 608 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:12:19.485215 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba5bdc3-9e0a-4dee-b9e6-f258138c2b91 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:19.587542 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:19.595151 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bbffa7-37a5-471e-9516-5defecc3843d None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:19.643361 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440774e-2500-44a8-91ad-c0126c5448a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:19.644316 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440774e-2500-44a8-91ad-c0126c5448a5 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:19.644775 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440774e-2500-44a8-91ad-c0126c5448a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:19.645270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440774e-2500-44a8-91ad-c0126c5448a5 None admin] PATH_INFO: `/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:19.647449 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b440774e-2500-44a8-91ad-c0126c5448a5 None admin] RBAC: Authorizing `identity:get_project(project_id=759a514c6fa54b4b8fa90f2d4f181f99)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:19.649650 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b440774e-2500-44a8-91ad-c0126c5448a5 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:19.652369 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 142/270] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:12:19 2022] GET /identity/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99 => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:19.653881 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1049e8-5e3a-49c0-b85c-121b001dca37 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:19.654320 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1049e8-5e3a-49c0-b85c-121b001dca37 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:19.654747 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1049e8-5e3a-49c0-b85c-121b001dca37 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:19.656616 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 143/271] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:19.665277 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e2d58-8c5e-4d24-809c-d078f594577d None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:19.665277 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e2d58-8c5e-4d24-809c-d078f594577d None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:19.665277 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e2d58-8c5e-4d24-809c-d078f594577d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:19.707617 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-a08e2d58-8c5e-4d24-809c-d078f594577d None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:19.710092 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-a08e2d58-8c5e-4d24-809c-d078f594577d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:19.742613 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba5bdc3-9e0a-4dee-b9e6-f258138c2b91 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:19.794806 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72aba5d-a9d8-4c97-83bd-e24390e6db77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:19.795446 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72aba5d-a9d8-4c97-83bd-e24390e6db77 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:19.795733 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72aba5d-a9d8-4c97-83bd-e24390e6db77 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:19.799011 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72aba5d-a9d8-4c97-83bd-e24390e6db77 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:19.799011 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72aba5d-a9d8-4c97-83bd-e24390e6db77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:19.799859 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72aba5d-a9d8-4c97-83bd-e24390e6db77 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:19.801573 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 129/272] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:12:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:19.802566 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72aba5d-a9d8-4c97-83bd-e24390e6db77 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:19.814671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 144/273] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:19.815940 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e2df7-95f9-4a01-b646-be274968f79f None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:19.819723 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e2df7-95f9-4a01-b646-be274968f79f None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:19.820274 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e2df7-95f9-4a01-b646-be274968f79f None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:19.821979 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 145/274] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:19.822876 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399e2df7-95f9-4a01-b646-be274968f79f None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:19.850511 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:19.929127 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72aba5d-a9d8-4c97-83bd-e24390e6db77 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:19.930809 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:19.941296 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399e2df7-95f9-4a01-b646-be274968f79f None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:19.964267 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf6c0d8-c48d-45e9-ab2d-6a381dd74fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:19.964969 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6c0d8-c48d-45e9-ab2d-6a381dd74fec None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:19.965163 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6c0d8-c48d-45e9-ab2d-6a381dd74fec None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:19.965442 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6c0d8-c48d-45e9-ab2d-6a381dd74fec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:19.966267 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf6c0d8-c48d-45e9-ab2d-6a381dd74fec None admin] RBAC: Adding query filter params (role.id=7c88c815db854954b516ab97cd471647, scope.project.id=759a514c6fa54b4b8fa90f2d4f181f99, user.id=f9487fe9b610445596a324d7de416e52) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:19.966582 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf6c0d8-c48d-45e9-ab2d-6a381dd74fec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:19.968039 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf6c0d8-c48d-45e9-ab2d-6a381dd74fec None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:19.973703 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 130/275] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:12:19 2022] GET /identity/v3/role_assignments?user.id=f9487fe9b610445596a324d7de416e52&scope.project.id=759a514c6fa54b4b8fa90f2d4f181f99&role.id=7c88c815db854954b516ab97cd471647 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:19.976043 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442890e1-7786-43ae-8d4b-a7c3e4aff71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:19.976684 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442890e1-7786-43ae-8d4b-a7c3e4aff71a None admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:19.976975 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442890e1-7786-43ae-8d4b-a7c3e4aff71a None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:19.977258 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442890e1-7786-43ae-8d4b-a7c3e4aff71a None admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:19.977952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442890e1-7786-43ae-8d4b-a7c3e4aff71a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:19.979418 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442890e1-7786-43ae-8d4b-a7c3e4aff71a None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:19.980999 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a11814-d2af-41ee-90f7-394d5a585fee None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:19.981295 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a11814-d2af-41ee-90f7-394d5a585fee None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:19.981582 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a11814-d2af-41ee-90f7-394d5a585fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:20.002370 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-c4a11814-d2af-41ee-90f7-394d5a585fee None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:20.003631 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-c4a11814-d2af-41ee-90f7-394d5a585fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:20.063762 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 146/276] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 5 15:12:19 2022] POST /identity/v3/users => generated 293 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:12:20.066218 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442890e1-7786-43ae-8d4b-a7c3e4aff71a None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:20.081610 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 131/277] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:20.093387 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898909d-3fd4-47cf-b6f0-c3798bdfad88 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:20.093681 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898909d-3fd4-47cf-b6f0-c3798bdfad88 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:20.093971 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898909d-3fd4-47cf-b6f0-c3798bdfad88 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:20.095430 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 132/278] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:20 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:20.103912 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8cd88c-35fa-48f5-93f5-78387c83907c None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:20.104180 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8cd88c-35fa-48f5-93f5-78387c83907c None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:20.104427 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8cd88c-35fa-48f5-93f5-78387c83907c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:20.130725 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-bb8cd88c-35fa-48f5-93f5-78387c83907c None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:20.132630 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-bb8cd88c-35fa-48f5-93f5-78387c83907c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:20.169794 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:20.228353 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442890e1-7786-43ae-8d4b-a7c3e4aff71a None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:20.233752 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 133/279] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 15:12:20.234762 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8cd88c-35fa-48f5-93f5-78387c83907c None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:20.275651 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528a583-f0d4-4e5f-a116-092bd28452f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:20.275651 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528a583-f0d4-4e5f-a116-092bd28452f1 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:20.275951 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528a583-f0d4-4e5f-a116-092bd28452f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:20.276200 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528a583-f0d4-4e5f-a116-092bd28452f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:20.289534 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0528a583-f0d4-4e5f-a116-092bd28452f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:20.296506 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-0528a583-f0d4-4e5f-a116-092bd28452f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 15:12:20.296506 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:20.296506 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:20.296506 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:20.296506 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:20.296506 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:20.296506 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:20.296506 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:20.296506 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:20.296506 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:20.296506 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:20.296506 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:20.296506 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:20.296506 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:20.296506 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:20.296506 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:20.297421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:20.297421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:20.297421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:20.297421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:20.297421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:20.297421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:20.297421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:20.297421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:20.297421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:20.297421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:20.297421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:20.297421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:20.297421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:20.297421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:20.297421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:20.297421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:20.298230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:20.298230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:20.298230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:20.298230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:20.298230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:20.298230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:20.298230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:20.298230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:20.298230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:20.298230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:20.298230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 15:12:20.298230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:20.298230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 147/280] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:12:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:20.299368 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528a583-f0d4-4e5f-a116-092bd28452f1 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:20.313555 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:20.368660 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8cd88c-35fa-48f5-93f5-78387c83907c None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:20.414283 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faccce9d-06f4-403f-b713-5fd7dc8c238d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:20.415311 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faccce9d-06f4-403f-b713-5fd7dc8c238d None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:20.415859 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faccce9d-06f4-403f-b713-5fd7dc8c238d None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:20.416453 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faccce9d-06f4-403f-b713-5fd7dc8c238d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:20.424399 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528a583-f0d4-4e5f-a116-092bd28452f1 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:20.431982 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faccce9d-06f4-403f-b713-5fd7dc8c238d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:20.439392 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-faccce9d-06f4-403f-b713-5fd7dc8c238d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 15:12:20.439392 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:20.439392 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:20.439392 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:20.439392 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:20.439392 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:20.439392 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:20.439392 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:20.439392 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:20.439392 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:20.439392 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:20.439392 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:20.439392 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:20.439392 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:20.439392 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:20.439392 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:20.440304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:20.440304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:20.440304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:20.440304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:20.440304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:20.440304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:20.440304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:20.440304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:20.440304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:20.440304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:20.440304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:20.440304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:20.440304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:20.440304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:20.440304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:20.440304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:20.441113 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:20.441113 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:20.441113 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:20.441113 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:20.441113 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:20.441113 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:20.441113 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:20.441113 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:20.441113 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:20.441113 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:20.441113 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 15:12:20.441113 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:20.443508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 134/281] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:12:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:20.445371 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faccce9d-06f4-403f-b713-5fd7dc8c238d None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:20.466587 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96c293a-2ad9-49ba-b695-b3b172143377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:20.467296 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c293a-2ad9-49ba-b695-b3b172143377 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:20.467589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c293a-2ad9-49ba-b695-b3b172143377 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:20.467871 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c293a-2ad9-49ba-b695-b3b172143377 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:20.477888 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:20.485110 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96c293a-2ad9-49ba-b695-b3b172143377 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:20.493208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-f96c293a-2ad9-49ba-b695-b3b172143377 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 15:12:20.493208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:20.493208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:20.493208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:20.493208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:20.493208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:20.493208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:20.493208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:20.493208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:20.493208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:20.493208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:20.493208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:20.493208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:20.493208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:20.493208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:20.493208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:20.494072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:20.494072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:20.494072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:20.494072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:20.494072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:20.494072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:20.494072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:20.494072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:20.494072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:20.494072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:20.494072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:20.494072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:20.494072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:20.494072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:20.494072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:20.494072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:20.494853 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:20.494853 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:20.494853 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:20.494853 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:20.494853 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:20.494853 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:20.494853 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:20.494853 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:20.494853 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:20.494853 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:20.494853 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 15:12:20.494853 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:20.494853 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 148/282] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:12:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:20.495490 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96c293a-2ad9-49ba-b695-b3b172143377 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:20.569905 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faccce9d-06f4-403f-b713-5fd7dc8c238d None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:20.580446 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:20.617812 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96c293a-2ad9-49ba-b695-b3b172143377 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:20.656222 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c1367-66da-4956-bc01-94b5da1cf970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:20.657252 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c1367-66da-4956-bc01-94b5da1cf970 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:20.657252 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c1367-66da-4956-bc01-94b5da1cf970 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:20.657437 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c1367-66da-4956-bc01-94b5da1cf970 None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:20.658245 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c1367-66da-4956-bc01-94b5da1cf970 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:20.658931 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c1367-66da-4956-bc01-94b5da1cf970 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:20.660193 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c1367-66da-4956-bc01-94b5da1cf970 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:20.672090 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 149/283] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 15:12:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:20.672265 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da1753-24f9-4021-89ea-655ecf05b0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:20.672382 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c1367-66da-4956-bc01-94b5da1cf970 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:20.672458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da1753-24f9-4021-89ea-655ecf05b0d6 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:20.672458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da1753-24f9-4021-89ea-655ecf05b0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:20.672458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da1753-24f9-4021-89ea-655ecf05b0d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:20.672458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38da1753-24f9-4021-89ea-655ecf05b0d6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:20.672458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38da1753-24f9-4021-89ea-655ecf05b0d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:20.677763 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38da1753-24f9-4021-89ea-655ecf05b0d6 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:20.694066 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 135/284] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 15:12:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:20.703684 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da1753-24f9-4021-89ea-655ecf05b0d6 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:20.781889 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c1367-66da-4956-bc01-94b5da1cf970 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:20.854121 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:20.867555 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:20.870547 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da1753-24f9-4021-89ea-655ecf05b0d6 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:20.877013 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2eb0f-bb3e-4cd1-bc0b-f6fc4ee8d38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:20.879238 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2eb0f-bb3e-4cd1-bc0b-f6fc4ee8d38a None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:20.879951 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2eb0f-bb3e-4cd1-bc0b-f6fc4ee8d38a None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:20.880445 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2eb0f-bb3e-4cd1-bc0b-f6fc4ee8d38a None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:20.882411 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a2eb0f-bb3e-4cd1-bc0b-f6fc4ee8d38a None admin] RBAC: Adding query filter params (name=member) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:20.883577 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a2eb0f-bb3e-4cd1-bc0b-f6fc4ee8d38a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:20.885360 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a2eb0f-bb3e-4cd1-bc0b-f6fc4ee8d38a None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:20.906796 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 150/285] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 15:12:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:20.910130 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2eb0f-bb3e-4cd1-bc0b-f6fc4ee8d38a None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:20.947186 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd7171-a98b-4fbc-8f73-4343b99470de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:20.949337 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd7171-a98b-4fbc-8f73-4343b99470de None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:20.949632 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd7171-a98b-4fbc-8f73-4343b99470de None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:20.949909 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd7171-a98b-4fbc-8f73-4343b99470de None admin] PATH_INFO: `/v3/users/edb7b556ac304a179ea214a93a59e188` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:20.955455 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd7171-a98b-4fbc-8f73-4343b99470de None admin] RBAC: Authorizing `identity:get_user(user_id=edb7b556ac304a179ea214a93a59e188)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:20.957265 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd7171-a98b-4fbc-8f73-4343b99470de None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:20.964084 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 136/286] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:12:20 2022] GET /identity/v3/users/edb7b556ac304a179ea214a93a59e188 => generated 297 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:20.965645 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd7171-a98b-4fbc-8f73-4343b99470de None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:21.047778 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2eb0f-bb3e-4cd1-bc0b-f6fc4ee8d38a None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:21.109036 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e39a0-6398-461a-a3b5-02a9a4e35285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:21.109701 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e39a0-6398-461a-a3b5-02a9a4e35285 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:21.109990 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e39a0-6398-461a-a3b5-02a9a4e35285 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:21.110272 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e39a0-6398-461a-a3b5-02a9a4e35285 None admin] PATH_INFO: `/v3/users/8efaf169967a4748804a261571549d98` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:21.129262 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505e39a0-6398-461a-a3b5-02a9a4e35285 None admin] RBAC: Authorizing `identity:get_user(user_id=8efaf169967a4748804a261571549d98)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:21.131968 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505e39a0-6398-461a-a3b5-02a9a4e35285 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:21.133784 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 151/287] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:12:20 2022] GET /identity/v3/users/8efaf169967a4748804a261571549d98 => generated 293 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:21.136205 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e39a0-6398-461a-a3b5-02a9a4e35285 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:21.159511 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd7171-a98b-4fbc-8f73-4343b99470de None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:21.196534 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:21.206411 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b93b26-a240-46ce-92de-a4210f2b2f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:21.207329 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b93b26-a240-46ce-92de-a4210f2b2f39 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:21.207624 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b93b26-a240-46ce-92de-a4210f2b2f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:21.207910 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b93b26-a240-46ce-92de-a4210f2b2f39 None admin] PATH_INFO: `/v3/users/f87a2184ca15477f95e0693ad00ae819` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:21.209878 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b93b26-a240-46ce-92de-a4210f2b2f39 None admin] RBAC: Authorizing `identity:get_user(user_id=f87a2184ca15477f95e0693ad00ae819)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:21.211927 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b93b26-a240-46ce-92de-a4210f2b2f39 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:21.213568 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 137/288] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:12:20 2022] GET /identity/v3/users/f87a2184ca15477f95e0693ad00ae819 => generated 297 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:21.214494 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b93b26-a240-46ce-92de-a4210f2b2f39 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:21.238206 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:21.257372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e39a0-6398-461a-a3b5-02a9a4e35285 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:21.300208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84899dba-d3be-47ad-98ce-1544d9d21603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:21.301209 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84899dba-d3be-47ad-98ce-1544d9d21603 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:21.301684 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84899dba-d3be-47ad-98ce-1544d9d21603 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:21.302152 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84899dba-d3be-47ad-98ce-1544d9d21603 None admin] PATH_INFO: `/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:21.304208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84899dba-d3be-47ad-98ce-1544d9d21603 None admin] RBAC: Authorizing `identity:get_project(project_id=759a514c6fa54b4b8fa90f2d4f181f99)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:21.306009 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84899dba-d3be-47ad-98ce-1544d9d21603 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:21.307936 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 152/289] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:12:21 2022] GET /identity/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:21.308899 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84899dba-d3be-47ad-98ce-1544d9d21603 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:21.336173 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b93b26-a240-46ce-92de-a4210f2b2f39 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:21.383094 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b1586-fac2-4f53-9521-85f87d1ea203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:21.384354 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b1586-fac2-4f53-9521-85f87d1ea203 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:21.384848 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b1586-fac2-4f53-9521-85f87d1ea203 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:21.385317 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b1586-fac2-4f53-9521-85f87d1ea203 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:21.386324 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330b1586-fac2-4f53-9521-85f87d1ea203 None admin] RBAC: Adding query filter params (role.id=d6c91838e4944cfbb23e1b6ecb009d26, user.id=8efaf169967a4748804a261571549d98, scope.system=all) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:21.386856 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330b1586-fac2-4f53-9521-85f87d1ea203 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:21.388662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330b1586-fac2-4f53-9521-85f87d1ea203 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:21.400420 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 138/290] 10.222.0.40 () {56 vars in 1308 bytes} [Fri Aug 5 15:12:21 2022] GET /identity/v3/role_assignments?user.id=8efaf169967a4748804a261571549d98&scope.system=all&role.id=d6c91838e4944cfbb23e1b6ecb009d26 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:21.401701 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b1586-fac2-4f53-9521-85f87d1ea203 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:21.418543 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84899dba-d3be-47ad-98ce-1544d9d21603 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:21.445260 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:21.468185 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a007ce-b0eb-4f93-be26-85df275f8fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:21.469101 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a007ce-b0eb-4f93-be26-85df275f8fad None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:21.470182 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a007ce-b0eb-4f93-be26-85df275f8fad None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:21.470636 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a007ce-b0eb-4f93-be26-85df275f8fad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:21.562150 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a007ce-b0eb-4f93-be26-85df275f8fad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:21.563805 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:21.565324 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a007ce-b0eb-4f93-be26-85df275f8fad None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:21.584998 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b1586-fac2-4f53-9521-85f87d1ea203 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:21.647708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d7eb1-3c1c-43b9-a583-aaf2f36b0f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:21.648915 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d7eb1-3c1c-43b9-a583-aaf2f36b0f92 None admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:21.648915 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d7eb1-3c1c-43b9-a583-aaf2f36b0f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:21.649317 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d7eb1-3c1c-43b9-a583-aaf2f36b0f92 None admin] PATH_INFO: `/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99/users/edb7b556ac304a179ea214a93a59e188/roles/09be81c3953242a08b57fd702a5082e3` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:21.653676 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3d7eb1-3c1c-43b9-a583-aaf2f36b0f92 None admin] RBAC: Authorizing `identity:create_grant(project_id=759a514c6fa54b4b8fa90f2d4f181f99, user_id=edb7b556ac304a179ea214a93a59e188, role_id=09be81c3953242a08b57fd702a5082e3)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:21.655298 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3d7eb1-3c1c-43b9-a583-aaf2f36b0f92 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:21.694098 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 153/291] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:12:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 15:12:21.699647 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a007ce-b0eb-4f93-be26-85df275f8fad None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:21.744443 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:21.746376 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 139/292] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 15:12:21 2022] PUT /identity/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99/users/edb7b556ac304a179ea214a93a59e188/roles/09be81c3953242a08b57fd702a5082e3 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:12:21.810681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a007ce-b0eb-4f93-be26-85df275f8fad None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:21.849062 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad4de2-d7f7-49b2-8671-3199cb51d881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:21.849756 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad4de2-d7f7-49b2-8671-3199cb51d881 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:21.850047 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad4de2-d7f7-49b2-8671-3199cb51d881 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:21.850390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad4de2-d7f7-49b2-8671-3199cb51d881 None admin] PATH_INFO: `/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:21.851878 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ad4de2-d7f7-49b2-8671-3199cb51d881 None admin] RBAC: Authorizing `identity:get_project(project_id=759a514c6fa54b4b8fa90f2d4f181f99)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:21.853432 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ad4de2-d7f7-49b2-8671-3199cb51d881 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:21.860244 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 154/293] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:12:21 2022] GET /identity/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:21.861763 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d7eb1-3c1c-43b9-a583-aaf2f36b0f92 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:21.995673 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d7eb1-3c1c-43b9-a583-aaf2f36b0f92 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:22.032942 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaa9ffb-fa84-49d7-8b0a-5c72d08c9c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:22.033570 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa9ffb-fa84-49d7-8b0a-5c72d08c9c52 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:22.033859 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa9ffb-fa84-49d7-8b0a-5c72d08c9c52 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:22.034195 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa9ffb-fa84-49d7-8b0a-5c72d08c9c52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:22.035040 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaa9ffb-fa84-49d7-8b0a-5c72d08c9c52 None admin] RBAC: Adding query filter params (role.id=d6c91838e4944cfbb23e1b6ecb009d26, scope.project.id=759a514c6fa54b4b8fa90f2d4f181f99, user.id=f87a2184ca15477f95e0693ad00ae819) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:22.035322 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaa9ffb-fa84-49d7-8b0a-5c72d08c9c52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:22.036867 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaa9ffb-fa84-49d7-8b0a-5c72d08c9c52 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:22.043559 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 140/294] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:12:21 2022] GET /identity/v3/role_assignments?user.id=f87a2184ca15477f95e0693ad00ae819&scope.project.id=759a514c6fa54b4b8fa90f2d4f181f99&role.id=d6c91838e4944cfbb23e1b6ecb009d26 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:22.489991 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22cf258-a0ed-479a-b5a2-bc9b5699a6e3 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:22.491051 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22cf258-a0ed-479a-b5a2-bc9b5699a6e3 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:22.491508 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22cf258-a0ed-479a-b5a2-bc9b5699a6e3 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:22.493432 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 155/295] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:22.499170 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625073de-6216-48d1-ad96-9f1fb16200e0 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:22.499730 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625073de-6216-48d1-ad96-9f1fb16200e0 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:22.500019 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625073de-6216-48d1-ad96-9f1fb16200e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:22.520006 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:22.529968 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0972b7d-0a6d-4e84-b6c1-88ac389e71d5 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:22.530246 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-625073de-6216-48d1-ad96-9f1fb16200e0 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:22.530360 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0972b7d-0a6d-4e84-b6c1-88ac389e71d5 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:22.530527 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0972b7d-0a6d-4e84-b6c1-88ac389e71d5 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:22.531933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 156/296] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 15:12:22.532186 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-625073de-6216-48d1-ad96-9f1fb16200e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:22.538964 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b152f1-bd98-4077-a2ae-b4e441d64d54 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:22.539261 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b152f1-bd98-4077-a2ae-b4e441d64d54 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:22.539546 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b152f1-bd98-4077-a2ae-b4e441d64d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:22.561730 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-85b152f1-bd98-4077-a2ae-b4e441d64d54 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:22.563937 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-85b152f1-bd98-4077-a2ae-b4e441d64d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:22.629377 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 141/297] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 15:12:22.653215 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625073de-6216-48d1-ad96-9f1fb16200e0 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:22.656529 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 157/298] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 15:12:22.681987 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b152f1-bd98-4077-a2ae-b4e441d64d54 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:22.701257 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:22.759086 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625073de-6216-48d1-ad96-9f1fb16200e0 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:22.792543 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b152f1-bd98-4077-a2ae-b4e441d64d54 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:22.794786 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec694e9f-52ed-4047-8949-3dd4292b8ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:22.795443 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec694e9f-52ed-4047-8949-3dd4292b8ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:22.795731 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec694e9f-52ed-4047-8949-3dd4292b8ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:22.796011 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec694e9f-52ed-4047-8949-3dd4292b8ff4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:22.797204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec694e9f-52ed-4047-8949-3dd4292b8ff4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:22.798656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec694e9f-52ed-4047-8949-3dd4292b8ff4 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:22.804139 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 142/299] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:12:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:12:22.806987 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec694e9f-52ed-4047-8949-3dd4292b8ff4 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:22.810705 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:22.836253 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bb71f3-c9dd-4040-8e0a-b69b55612275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:22.837344 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb71f3-c9dd-4040-8e0a-b69b55612275 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:22.837825 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb71f3-c9dd-4040-8e0a-b69b55612275 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:22.838490 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb71f3-c9dd-4040-8e0a-b69b55612275 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:22.852421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bb71f3-c9dd-4040-8e0a-b69b55612275 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:22.858603 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-f2bb71f3-c9dd-4040-8e0a-b69b55612275 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 15:12:22.858603 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:22.858603 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:22.858603 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:22.858603 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:22.858603 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:22.858603 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:22.858603 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:22.858603 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:22.858603 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:22.858603 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:22.858603 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:22.858603 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:22.858603 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:22.858603 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:22.858603 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:22.859463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:22.859463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:22.859463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:22.859463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:22.859463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:22.859463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:22.859463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:22.859463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:22.859463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:22.859463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:22.859463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:22.859463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:22.859463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:22.859463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:22.859463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:22.859463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:22.860679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:22.860679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:22.860679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:22.860679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:22.860679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:22.860679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:22.860679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:22.860679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:22.860679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:22.860679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:22.860679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 15:12:22.860679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:22.862904 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 158/300] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:12:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:12:22.868710 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bb71f3-c9dd-4040-8e0a-b69b55612275 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:22.915574 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec694e9f-52ed-4047-8949-3dd4292b8ff4 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:22.949479 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9740742-b98f-4787-ae42-05b2eb1af990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:22.950105 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9740742-b98f-4787-ae42-05b2eb1af990 None admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:22.950396 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9740742-b98f-4787-ae42-05b2eb1af990 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:22.950676 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9740742-b98f-4787-ae42-05b2eb1af990 None admin] PATH_INFO: `/v3/groups` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:22.951346 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9740742-b98f-4787-ae42-05b2eb1af990 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:22.952823 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9740742-b98f-4787-ae42-05b2eb1af990 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:22.955406 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:22.979730 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bb71f3-c9dd-4040-8e0a-b69b55612275 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:23.015339 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfe3891-a3b8-47c9-bd82-0be877378a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:23.015948 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe3891-a3b8-47c9-bd82-0be877378a7b None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:23.016265 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe3891-a3b8-47c9-bd82-0be877378a7b None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:23.016542 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe3891-a3b8-47c9-bd82-0be877378a7b None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:23.017256 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfe3891-a3b8-47c9-bd82-0be877378a7b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:23.017562 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfe3891-a3b8-47c9-bd82-0be877378a7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:23.018940 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfe3891-a3b8-47c9-bd82-0be877378a7b None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:23.019611 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 143/301] 10.222.0.40 () {60 vars in 1134 bytes} [Fri Aug 5 15:12:22 2022] POST /identity/v3/groups => generated 229 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:12:23.033029 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 159/302] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 15:12:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:23.039604 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9740742-b98f-4787-ae42-05b2eb1af990 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:23.111815 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:23.155022 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9740742-b98f-4787-ae42-05b2eb1af990 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:23.196591 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334949f-bb74-4aae-868f-c9cb0d58f8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:23.197253 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334949f-bb74-4aae-868f-c9cb0d58f8d1 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:23.197540 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334949f-bb74-4aae-868f-c9cb0d58f8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:23.197821 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334949f-bb74-4aae-868f-c9cb0d58f8d1 None admin] PATH_INFO: `/v3/users/b5c970ee793a40febb1fb633ab396773` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:23.213978 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9334949f-bb74-4aae-868f-c9cb0d58f8d1 None admin] RBAC: Authorizing `identity:get_user(user_id=b5c970ee793a40febb1fb633ab396773)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:23.215475 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9334949f-bb74-4aae-868f-c9cb0d58f8d1 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:23.217542 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 144/303] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:12:23 2022] GET /identity/v3/users/b5c970ee793a40febb1fb633ab396773 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:23.223374 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfe3891-a3b8-47c9-bd82-0be877378a7b None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:23.340330 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfe3891-a3b8-47c9-bd82-0be877378a7b None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:23.378761 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf1f1a0-bacd-4dee-91d6-c74dddba0919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:23.379394 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1f1a0-bacd-4dee-91d6-c74dddba0919 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:23.379678 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1f1a0-bacd-4dee-91d6-c74dddba0919 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:23.379955 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1f1a0-bacd-4dee-91d6-c74dddba0919 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:23.381369 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf1f1a0-bacd-4dee-91d6-c74dddba0919 None admin] RBAC: Adding query filter params (role.id=09be81c3953242a08b57fd702a5082e3, user.id=b5c970ee793a40febb1fb633ab396773, scope.system=all) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:23.381687 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf1f1a0-bacd-4dee-91d6-c74dddba0919 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:23.383114 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf1f1a0-bacd-4dee-91d6-c74dddba0919 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:23.388745 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 160/304] 10.222.0.40 () {56 vars in 1308 bytes} [Fri Aug 5 15:12:23 2022] GET /identity/v3/role_assignments?user.id=b5c970ee793a40febb1fb633ab396773&scope.system=all&role.id=09be81c3953242a08b57fd702a5082e3 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:23.609923 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f710d11-7a07-474a-9d40-f4f52635c5ed None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:23.610289 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f710d11-7a07-474a-9d40-f4f52635c5ed None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:23.610503 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f710d11-7a07-474a-9d40-f4f52635c5ed None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:23.611792 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 145/305] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:23.617462 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e6dc1-3a4b-44c3-bcc1-276f0e60dd08 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:23.617755 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e6dc1-3a4b-44c3-bcc1-276f0e60dd08 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:23.618035 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e6dc1-3a4b-44c3-bcc1-276f0e60dd08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:23.625021 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:23.637639 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-a55e6dc1-3a4b-44c3-bcc1-276f0e60dd08 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:23.638935 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-a55e6dc1-3a4b-44c3-bcc1-276f0e60dd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:23.706383 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 161/306] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:23.727945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f710d11-7a07-474a-9d40-f4f52635c5ed None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:23.840397 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f710d11-7a07-474a-9d40-f4f52635c5ed None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:23.863955 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:23.877024 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12783317-90b2-483e-9bad-42f4cc3993d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:23.877664 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12783317-90b2-483e-9bad-42f4cc3993d0 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:23.877947 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12783317-90b2-483e-9bad-42f4cc3993d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:23.878224 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12783317-90b2-483e-9bad-42f4cc3993d0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:23.889274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12783317-90b2-483e-9bad-42f4cc3993d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:23.894330 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-12783317-90b2-483e-9bad-42f4cc3993d0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 15:12:23.894330 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:23.894330 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:23.894330 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:23.894330 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:23.894330 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:23.894330 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:23.894330 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:23.894330 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:23.894330 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:23.894330 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:23.894330 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:23.894330 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:23.894330 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:23.894330 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:23.894330 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:23.895223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:23.895223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:23.895223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:23.895223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:23.895223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:23.895223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:23.895223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:23.895223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:23.895223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:23.895223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:23.895223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:23.895223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:23.895223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:23.895223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:23.895223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:23.895223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:23.896033 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:23.896033 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:23.896033 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:23.896033 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:23.896033 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:23.896033 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:23.896033 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:23.896033 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:23.896033 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:23.896033 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:23.896033 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 15:12:23.896033 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:23.896033 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 146/307] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:12:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:23.901354 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e6dc1-3a4b-44c3-bcc1-276f0e60dd08 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:24.006977 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e6dc1-3a4b-44c3-bcc1-276f0e60dd08 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:24.036560 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:24.042073 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2960177a-00c3-47f1-bd12-2b58d9e09ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:24.042685 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960177a-00c3-47f1-bd12-2b58d9e09ec9 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:24.042963 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960177a-00c3-47f1-bd12-2b58d9e09ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:24.043238 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960177a-00c3-47f1-bd12-2b58d9e09ec9 None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:24.043952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2960177a-00c3-47f1-bd12-2b58d9e09ec9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:24.044295 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2960177a-00c3-47f1-bd12-2b58d9e09ec9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:24.045656 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2960177a-00c3-47f1-bd12-2b58d9e09ec9 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:24.054888 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 162/308] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 15:12:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:24.060344 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12783317-90b2-483e-9bad-42f4cc3993d0 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:24.167523 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12783317-90b2-483e-9bad-42f4cc3993d0 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:24.203545 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4dcc60-72ec-4e69-9a4e-52d585de5e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:24.204220 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4dcc60-72ec-4e69-9a4e-52d585de5e8d None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:24.204572 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4dcc60-72ec-4e69-9a4e-52d585de5e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:24.204852 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4dcc60-72ec-4e69-9a4e-52d585de5e8d None admin] PATH_INFO: `/v3/users/8efaf169967a4748804a261571549d98` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:24.206042 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4dcc60-72ec-4e69-9a4e-52d585de5e8d None admin] RBAC: Authorizing `identity:get_user(user_id=8efaf169967a4748804a261571549d98)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:24.207502 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4dcc60-72ec-4e69-9a4e-52d585de5e8d None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:24.209485 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 147/309] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:12:24 2022] GET /identity/v3/users/8efaf169967a4748804a261571549d98 => generated 293 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:24.214784 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2960177a-00c3-47f1-bd12-2b58d9e09ec9 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:24.276302 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c04f6f-7925-4dcf-8b08-553a20f3a140 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:24.276608 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c04f6f-7925-4dcf-8b08-553a20f3a140 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:24.276861 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c04f6f-7925-4dcf-8b08-553a20f3a140 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:24.278198 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 148/310] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 15:12:24.283236 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc9d24-ce50-404d-93fb-2b3122b9d99b None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:24.283520 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc9d24-ce50-404d-93fb-2b3122b9d99b None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:24.283794 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc9d24-ce50-404d-93fb-2b3122b9d99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:24.307963 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-f7fc9d24-ce50-404d-93fb-2b3122b9d99b None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:24.309293 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-f7fc9d24-ce50-404d-93fb-2b3122b9d99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:24.326425 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2960177a-00c3-47f1-bd12-2b58d9e09ec9 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:24.363501 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782d4fa-268b-4d26-b854-c210d6edb3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:24.364240 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782d4fa-268b-4d26-b854-c210d6edb3f2 None admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:24.364523 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782d4fa-268b-4d26-b854-c210d6edb3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:24.364877 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782d4fa-268b-4d26-b854-c210d6edb3f2 None admin] PATH_INFO: `/v3/system/users/8efaf169967a4748804a261571549d98/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:24.366354 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4782d4fa-268b-4d26-b854-c210d6edb3f2 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8efaf169967a4748804a261571549d98, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:24.367723 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4782d4fa-268b-4d26-b854-c210d6edb3f2 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:24.369914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:24.392145 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 149/311] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 15:12:24.416883 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc9d24-ce50-404d-93fb-2b3122b9d99b None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:24.469426 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 163/312] 10.222.0.40 () {58 vars in 1334 bytes} [Fri Aug 5 15:12:24 2022] PUT /identity/v3/system/users/8efaf169967a4748804a261571549d98/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:12:24.544766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc9d24-ce50-404d-93fb-2b3122b9d99b None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:24.584798 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb1eecb-3d81-45ad-a1d6-edf9869241b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:24.585475 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb1eecb-3d81-45ad-a1d6-edf9869241b1 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:24.585741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb1eecb-3d81-45ad-a1d6-edf9869241b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:24.586020 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb1eecb-3d81-45ad-a1d6-edf9869241b1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:24.598250 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb1eecb-3d81-45ad-a1d6-edf9869241b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:24.604015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-0bb1eecb-3d81-45ad-a1d6-edf9869241b1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 15:12:24.604015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:24.604015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:24.604015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:24.604015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:24.604015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:24.604015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:24.604015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:24.604015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:24.604015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:24.604015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:24.604015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:24.604015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:24.604015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:24.604015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:24.604015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:24.604968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:24.604968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:24.604968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:24.604968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:24.604968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:24.604968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:24.604968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:24.604968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:24.604968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:24.604968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:24.604968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:24.604968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:24.604968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:24.604968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:24.604968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:24.604968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:24.605746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:24.605746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:24.605746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:24.605746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:24.605746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:24.605746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:24.605746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:24.605746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:24.605746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:24.605746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:24.605746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 15:12:24.605746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:24.606342 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 150/313] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:12:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:12:24.611670 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782d4fa-268b-4d26-b854-c210d6edb3f2 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:24.681122 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:24.729202 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782d4fa-268b-4d26-b854-c210d6edb3f2 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:24.765333 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0e70fa-e564-4e00-bbcb-6c520b417a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:24.766456 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e70fa-e564-4e00-bbcb-6c520b417a91 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:24.766957 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e70fa-e564-4e00-bbcb-6c520b417a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:24.767445 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e70fa-e564-4e00-bbcb-6c520b417a91 None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:24.768379 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0e70fa-e564-4e00-bbcb-6c520b417a91 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:24.768880 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0e70fa-e564-4e00-bbcb-6c520b417a91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:24.770506 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0e70fa-e564-4e00-bbcb-6c520b417a91 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:24.780397 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 164/314] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 15:12:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:24.792104 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb1eecb-3d81-45ad-a1d6-edf9869241b1 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:24.837771 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:24.899009 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb1eecb-3d81-45ad-a1d6-edf9869241b1 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:24.934591 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e9ba44-efaf-4fff-a3c5-4ca7a6dd43e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:24.935483 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9ba44-efaf-4fff-a3c5-4ca7a6dd43e8 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:24.935935 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9ba44-efaf-4fff-a3c5-4ca7a6dd43e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:24.936399 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9ba44-efaf-4fff-a3c5-4ca7a6dd43e8 None admin] PATH_INFO: `/v3/users/edb7b556ac304a179ea214a93a59e188` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:24.937798 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e9ba44-efaf-4fff-a3c5-4ca7a6dd43e8 None admin] RBAC: Authorizing `identity:get_user(user_id=edb7b556ac304a179ea214a93a59e188)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:24.939390 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e9ba44-efaf-4fff-a3c5-4ca7a6dd43e8 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:24.941537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 151/315] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:12:24 2022] GET /identity/v3/users/edb7b556ac304a179ea214a93a59e188 => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:24.947803 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0e70fa-e564-4e00-bbcb-6c520b417a91 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:25.051061 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0e70fa-e564-4e00-bbcb-6c520b417a91 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:25.072179 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:25.086218 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d96a09-c32b-4a10-a52e-fc2e3e320f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:25.086842 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d96a09-c32b-4a10-a52e-fc2e3e320f5a None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:25.087126 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d96a09-c32b-4a10-a52e-fc2e3e320f5a None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:25.087399 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d96a09-c32b-4a10-a52e-fc2e3e320f5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:25.156669 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d96a09-c32b-4a10-a52e-fc2e3e320f5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:25.158146 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d96a09-c32b-4a10-a52e-fc2e3e320f5a None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:25.242740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb2667-c478-4063-9cb8-89a11f6e5aa2 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:25.243282 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb2667-c478-4063-9cb8-89a11f6e5aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:25.243282 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb2667-c478-4063-9cb8-89a11f6e5aa2 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:25.250406 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 152/316] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:25.252619 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a824d5-c12d-499f-a140-1415c361ded6 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:25.253019 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a824d5-c12d-499f-a140-1415c361ded6 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:25.253404 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a824d5-c12d-499f-a140-1415c361ded6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:25.260046 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 165/317] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:12:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 15:12:25.265517 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d96a09-c32b-4a10-a52e-fc2e3e320f5a None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:25.279474 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-a7a824d5-c12d-499f-a140-1415c361ded6 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:25.280720 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-a7a824d5-c12d-499f-a140-1415c361ded6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:25.291111 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:25.381359 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d96a09-c32b-4a10-a52e-fc2e3e320f5a None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:25.381740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 153/318] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 15:12:25.404099 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a824d5-c12d-499f-a140-1415c361ded6 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:25.420772 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e126a89-045d-4a37-8bde-839a01673233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:25.421409 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e126a89-045d-4a37-8bde-839a01673233 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:25.421692 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e126a89-045d-4a37-8bde-839a01673233 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:25.422020 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e126a89-045d-4a37-8bde-839a01673233 None admin] PATH_INFO: `/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:25.423338 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e126a89-045d-4a37-8bde-839a01673233 None admin] RBAC: Authorizing `identity:get_project(project_id=759a514c6fa54b4b8fa90f2d4f181f99)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:25.424879 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e126a89-045d-4a37-8bde-839a01673233 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:25.426982 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 166/319] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:12:25 2022] GET /identity/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:25.433232 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e126a89-045d-4a37-8bde-839a01673233 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:25.448590 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:25.521214 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a824d5-c12d-499f-a140-1415c361ded6 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:25.535705 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:25.550201 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e126a89-045d-4a37-8bde-839a01673233 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:25.558455 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd81c391-defe-4813-bb8d-b31c8094808c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:25.559384 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81c391-defe-4813-bb8d-b31c8094808c None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:25.559900 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81c391-defe-4813-bb8d-b31c8094808c None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:25.560359 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81c391-defe-4813-bb8d-b31c8094808c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:25.561654 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd81c391-defe-4813-bb8d-b31c8094808c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:25.563228 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd81c391-defe-4813-bb8d-b31c8094808c None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:25.565400 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 154/320] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:12:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:25.566673 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893fb6c-e369-4473-89e1-8950e66f3bac None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:25.571114 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893fb6c-e369-4473-89e1-8950e66f3bac None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:25.571114 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893fb6c-e369-4473-89e1-8950e66f3bac None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:25.571114 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 155/321] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:25.574795 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c1fae-478f-4156-8bfd-5743ce6b6622 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:25.575084 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c1fae-478f-4156-8bfd-5743ce6b6622 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:25.575361 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c1fae-478f-4156-8bfd-5743ce6b6622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:25.590460 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5de5c3-25c0-4c1a-abd7-c50f119a6168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:25.591077 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5de5c3-25c0-4c1a-abd7-c50f119a6168 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:25.591363 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5de5c3-25c0-4c1a-abd7-c50f119a6168 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:25.591642 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5de5c3-25c0-4c1a-abd7-c50f119a6168 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:25.592478 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5de5c3-25c0-4c1a-abd7-c50f119a6168 None admin] RBAC: Adding query filter params (role.id=09be81c3953242a08b57fd702a5082e3, scope.project.id=759a514c6fa54b4b8fa90f2d4f181f99, user.id=edb7b556ac304a179ea214a93a59e188) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:25.592795 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5de5c3-25c0-4c1a-abd7-c50f119a6168 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:25.594235 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5de5c3-25c0-4c1a-abd7-c50f119a6168 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:25.595674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-784c1fae-478f-4156-8bfd-5743ce6b6622 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:25.597486 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-784c1fae-478f-4156-8bfd-5743ce6b6622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:25.600904 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 167/322] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:12:25 2022] GET /identity/v3/role_assignments?user.id=edb7b556ac304a179ea214a93a59e188&scope.project.id=759a514c6fa54b4b8fa90f2d4f181f99&role.id=09be81c3953242a08b57fd702a5082e3 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:12:25.601609 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5de5c3-25c0-4c1a-abd7-c50f119a6168 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:25.692581 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 156/323] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:25.709253 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c1fae-478f-4156-8bfd-5743ce6b6622 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:25.735074 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5de5c3-25c0-4c1a-abd7-c50f119a6168 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:25.751894 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:25.785046 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8560bd-06be-4371-a5e8-dfff1208e43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:25.785668 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8560bd-06be-4371-a5e8-dfff1208e43c None admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:25.785952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8560bd-06be-4371-a5e8-dfff1208e43c None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:25.786230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8560bd-06be-4371-a5e8-dfff1208e43c None admin] PATH_INFO: `/v3/groups` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:25.788393 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8560bd-06be-4371-a5e8-dfff1208e43c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:25.790080 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8560bd-06be-4371-a5e8-dfff1208e43c None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:25.832750 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c1fae-478f-4156-8bfd-5743ce6b6622 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:25.869964 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3f9427-f66b-4b05-8997-8a1cdf7e7389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:25.870614 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f9427-f66b-4b05-8997-8a1cdf7e7389 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:25.870903 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f9427-f66b-4b05-8997-8a1cdf7e7389 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:25.871189 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f9427-f66b-4b05-8997-8a1cdf7e7389 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:25.882298 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3f9427-f66b-4b05-8997-8a1cdf7e7389 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:25.886652 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 168/324] 10.222.0.40 () {60 vars in 1134 bytes} [Fri Aug 5 15:12:25 2022] POST /identity/v3/groups => generated 226 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:12:25.887365 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-bf3f9427-f66b-4b05-8997-8a1cdf7e7389 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 15:12:25.887365 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:25.887365 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:25.887365 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:25.887365 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:25.887365 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:25.887365 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:25.887365 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:25.887365 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:25.887365 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:25.887365 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:25.887365 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:25.887365 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:25.887365 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:25.887365 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:25.887365 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:25.889997 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:25.889997 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:25.889997 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:25.889997 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:25.889997 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:25.889997 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:25.889997 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:25.889997 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:25.889997 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:25.889997 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:25.889997 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:25.889997 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:25.889997 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:25.889997 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:25.889997 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:25.889997 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:25.890793 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:25.890793 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:25.890793 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:25.890793 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:25.890793 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:25.890793 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:25.890793 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:25.890793 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:25.890793 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:25.890793 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:25.890793 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 15:12:25.890793 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:25.890793 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 157/325] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:12:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:12:25.895070 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8560bd-06be-4371-a5e8-dfff1208e43c None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:25.913512 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:26.007353 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8560bd-06be-4371-a5e8-dfff1208e43c None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:26.044492 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5dc01f-26fc-434d-955a-a2db1ebe6955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:26.045125 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5dc01f-26fc-434d-955a-a2db1ebe6955 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:26.045406 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5dc01f-26fc-434d-955a-a2db1ebe6955 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:26.045688 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5dc01f-26fc-434d-955a-a2db1ebe6955 None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:26.046410 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5dc01f-26fc-434d-955a-a2db1ebe6955 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:26.046717 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5dc01f-26fc-434d-955a-a2db1ebe6955 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:26.048134 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5dc01f-26fc-434d-955a-a2db1ebe6955 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:26.057483 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 169/326] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 15:12:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:26.063103 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3f9427-f66b-4b05-8997-8a1cdf7e7389 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:26.168680 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:26.176421 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3f9427-f66b-4b05-8997-8a1cdf7e7389 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:26.211028 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3885b6b9-1b17-4c51-be06-e9a11ad78de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:26.211682 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885b6b9-1b17-4c51-be06-e9a11ad78de1 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:26.211962 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885b6b9-1b17-4c51-be06-e9a11ad78de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:26.212269 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885b6b9-1b17-4c51-be06-e9a11ad78de1 None admin] PATH_INFO: `/v3/users/b5c970ee793a40febb1fb633ab396773` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:26.213442 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3885b6b9-1b17-4c51-be06-e9a11ad78de1 None admin] RBAC: Authorizing `identity:get_user(user_id=b5c970ee793a40febb1fb633ab396773)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:26.214912 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3885b6b9-1b17-4c51-be06-e9a11ad78de1 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:26.216852 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 158/327] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:12:26 2022] GET /identity/v3/users/b5c970ee793a40febb1fb633ab396773 => generated 293 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:26.222355 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5dc01f-26fc-434d-955a-a2db1ebe6955 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:26.304405 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:26.330161 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5dc01f-26fc-434d-955a-a2db1ebe6955 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:26.369577 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeea20f-c26f-4a12-b9ad-3418629ab139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:26.370253 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeea20f-c26f-4a12-b9ad-3418629ab139 None admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:26.370541 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeea20f-c26f-4a12-b9ad-3418629ab139 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:26.370828 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeea20f-c26f-4a12-b9ad-3418629ab139 None admin] PATH_INFO: `/v3/system/users/b5c970ee793a40febb1fb633ab396773/roles/09be81c3953242a08b57fd702a5082e3` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:26.372364 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeea20f-c26f-4a12-b9ad-3418629ab139 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b5c970ee793a40febb1fb633ab396773, role_id=09be81c3953242a08b57fd702a5082e3)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:26.373759 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeea20f-c26f-4a12-b9ad-3418629ab139 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:26.435236 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 170/328] 10.222.0.40 () {58 vars in 1334 bytes} [Fri Aug 5 15:12:26 2022] PUT /identity/v3/system/users/b5c970ee793a40febb1fb633ab396773/roles/09be81c3953242a08b57fd702a5082e3 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:12:26.632500 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8cff6-f7f6-434c-8a85-79e51ad26a9e None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:26.632973 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8cff6-f7f6-434c-8a85-79e51ad26a9e None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:26.632973 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8cff6-f7f6-434c-8a85-79e51ad26a9e None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:26.634595 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 159/329] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:26.640023 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d4b63-e29f-4e8f-9808-85a8dab02f26 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:26.640409 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d4b63-e29f-4e8f-9808-85a8dab02f26 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:26.640695 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d4b63-e29f-4e8f-9808-85a8dab02f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:26.663327 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-b73d4b63-e29f-4e8f-9808-85a8dab02f26 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:26.664555 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-b73d4b63-e29f-4e8f-9808-85a8dab02f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:26.746375 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 171/330] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:26.781312 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a8cff6-f7f6-434c-8a85-79e51ad26a9e None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:26.923685 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a8cff6-f7f6-434c-8a85-79e51ad26a9e None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:26.966375 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c512d79c-aadd-4aef-97ee-e2bf3cdcfe9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:26.967650 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c512d79c-aadd-4aef-97ee-e2bf3cdcfe9e None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:26.968172 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c512d79c-aadd-4aef-97ee-e2bf3cdcfe9e None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:26.968689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c512d79c-aadd-4aef-97ee-e2bf3cdcfe9e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:26.985923 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c512d79c-aadd-4aef-97ee-e2bf3cdcfe9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:26.991759 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-c512d79c-aadd-4aef-97ee-e2bf3cdcfe9e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 15:12:26.991759 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:26.991759 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:26.991759 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:26.991759 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:26.991759 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:26.991759 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:26.991759 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:26.991759 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:26.991759 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:26.991759 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:26.991759 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:26.991759 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:26.991759 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:26.991759 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:26.991759 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:26.992795 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:26.992795 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:26.992795 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:26.992795 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:26.992795 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:26.992795 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:26.992795 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:26.992795 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:26.992795 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:26.992795 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:26.992795 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:26.992795 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:26.992795 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:26.992795 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:26.992795 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:26.992795 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:26.993782 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:26.993782 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:26.993782 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:26.993782 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:26.993782 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:26.993782 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:26.993782 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:26.993782 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:26.993782 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:26.993782 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:26.993782 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 15:12:26.993782 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:26.995773 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 160/331] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:12:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:27.002065 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d4b63-e29f-4e8f-9808-85a8dab02f26 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:27.096191 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:27.135044 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d4b63-e29f-4e8f-9808-85a8dab02f26 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:27.187536 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe6e24-e72d-4eb5-81f3-d1995a54ff5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:27.188166 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe6e24-e72d-4eb5-81f3-d1995a54ff5c None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:27.188392 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe6e24-e72d-4eb5-81f3-d1995a54ff5c None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:27.188674 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe6e24-e72d-4eb5-81f3-d1995a54ff5c None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:27.189415 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbe6e24-e72d-4eb5-81f3-d1995a54ff5c None admin] RBAC: Adding query filter params (name=member) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:27.189705 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbe6e24-e72d-4eb5-81f3-d1995a54ff5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:27.191100 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbe6e24-e72d-4eb5-81f3-d1995a54ff5c None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:27.205305 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 172/332] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 15:12:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:27.210722 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c512d79c-aadd-4aef-97ee-e2bf3cdcfe9e None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:27.244990 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:27.345526 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c512d79c-aadd-4aef-97ee-e2bf3cdcfe9e None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:27.392210 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd519946-c4ee-4025-8b84-50257bf8f4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:27.393249 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd519946-c4ee-4025-8b84-50257bf8f4aa None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:27.393769 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd519946-c4ee-4025-8b84-50257bf8f4aa None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:27.394224 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd519946-c4ee-4025-8b84-50257bf8f4aa None admin] PATH_INFO: `/v3/users/8efaf169967a4748804a261571549d98` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:27.400784 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd519946-c4ee-4025-8b84-50257bf8f4aa None admin] RBAC: Authorizing `identity:get_user(user_id=8efaf169967a4748804a261571549d98)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:27.402454 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd519946-c4ee-4025-8b84-50257bf8f4aa None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:27.404827 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 161/333] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:12:27 2022] GET /identity/v3/users/8efaf169967a4748804a261571549d98 => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:27.410360 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe6e24-e72d-4eb5-81f3-d1995a54ff5c None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:27.536178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe6e24-e72d-4eb5-81f3-d1995a54ff5c None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:27.569139 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:27.576989 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e3226-2584-4b67-ba6e-009a1c7ec12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:27.577617 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e3226-2584-4b67-ba6e-009a1c7ec12a None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:27.577902 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e3226-2584-4b67-ba6e-009a1c7ec12a None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:27.578185 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e3226-2584-4b67-ba6e-009a1c7ec12a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:27.578984 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2e3226-2584-4b67-ba6e-009a1c7ec12a None admin] RBAC: Adding query filter params (role.id=d6c91838e4944cfbb23e1b6ecb009d26, user.id=8efaf169967a4748804a261571549d98, scope.system=all) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:27.579299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2e3226-2584-4b67-ba6e-009a1c7ec12a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:27.584761 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2e3226-2584-4b67-ba6e-009a1c7ec12a None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:27.597875 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 173/334] 10.222.0.40 () {56 vars in 1308 bytes} [Fri Aug 5 15:12:27 2022] GET /identity/v3/role_assignments?user.id=8efaf169967a4748804a261571549d98&scope.system=all&role.id=d6c91838e4944cfbb23e1b6ecb009d26 => generated 510 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:28.633555 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a2dae-041f-4e07-a54b-1874f63e6433 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:28.634714 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a2dae-041f-4e07-a54b-1874f63e6433 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:28.635211 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a2dae-041f-4e07-a54b-1874f63e6433 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:28.637103 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 162/335] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:28.643882 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca996cd1-4a7d-41aa-8855-5d574af035a1 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:28.644754 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca996cd1-4a7d-41aa-8855-5d574af035a1 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:28.645240 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca996cd1-4a7d-41aa-8855-5d574af035a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:28.666373 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-ca996cd1-4a7d-41aa-8855-5d574af035a1 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:28.667926 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-ca996cd1-4a7d-41aa-8855-5d574af035a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:28.744338 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 174/336] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:28.766175 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a2dae-041f-4e07-a54b-1874f63e6433 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:28.882717 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:28.894397 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a2dae-041f-4e07-a54b-1874f63e6433 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:28.931114 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a5ddb2-fd78-4ed7-98b5-dac4137cfc73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:28.931828 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5ddb2-fd78-4ed7-98b5-dac4137cfc73 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:28.932139 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5ddb2-fd78-4ed7-98b5-dac4137cfc73 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:28.932422 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5ddb2-fd78-4ed7-98b5-dac4137cfc73 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:28.950005 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a5ddb2-fd78-4ed7-98b5-dac4137cfc73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:28.955185 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-b8a5ddb2-fd78-4ed7-98b5-dac4137cfc73 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 15:12:28.955185 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:28.955185 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:28.955185 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:28.955185 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:28.955185 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:28.955185 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:28.955185 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:28.955185 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:28.955185 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:28.955185 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:28.955185 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:28.955185 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:28.955185 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:28.955185 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:28.955185 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:28.956047 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:28.956047 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:28.956047 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:28.956047 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:28.956047 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:28.956047 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:28.956047 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:28.956047 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:28.956047 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:28.956047 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:28.956047 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:28.956047 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:28.956047 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:28.956047 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:28.956047 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:28.956047 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:28.956873 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:28.956873 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:28.956873 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:28.956873 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:28.956873 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:28.956873 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:28.956873 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:28.956873 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:28.956873 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:28.956873 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:28.956873 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 15:12:28.956873 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:28.959923 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 163/337] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:12:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:28.963436 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca996cd1-4a7d-41aa-8855-5d574af035a1 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:29.083832 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca996cd1-4a7d-41aa-8855-5d574af035a1 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:29.091997 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4881c7-52b2-49bc-b9c4-aa11664ec313 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:29.092377 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4881c7-52b2-49bc-b9c4-aa11664ec313 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:29.092670 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4881c7-52b2-49bc-b9c4-aa11664ec313 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:29.094034 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 164/338] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 15:12:29.099885 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678a624-885d-4ba7-807b-27da7feef111 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:29.100196 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678a624-885d-4ba7-807b-27da7feef111 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:29.100471 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678a624-885d-4ba7-807b-27da7feef111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:29.111248 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:29.128552 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-b678a624-885d-4ba7-807b-27da7feef111 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:29.129791 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-b678a624-885d-4ba7-807b-27da7feef111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:29.131562 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dee1a6b-6ccf-4bd4-bf9a-d982f21c8aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:29.132243 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee1a6b-6ccf-4bd4-bf9a-d982f21c8aec None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:29.132533 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee1a6b-6ccf-4bd4-bf9a-d982f21c8aec None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:29.132814 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee1a6b-6ccf-4bd4-bf9a-d982f21c8aec None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:29.133535 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dee1a6b-6ccf-4bd4-bf9a-d982f21c8aec None admin] RBAC: Adding query filter params (name=member) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:29.134017 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dee1a6b-6ccf-4bd4-bf9a-d982f21c8aec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:29.135392 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dee1a6b-6ccf-4bd4-bf9a-d982f21c8aec None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:29.151762 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 175/339] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 15:12:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:29.153420 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dee1a6b-6ccf-4bd4-bf9a-d982f21c8aec None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:29.275141 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 165/340] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:29.314541 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b678a624-885d-4ba7-807b-27da7feef111 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:29.339434 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dee1a6b-6ccf-4bd4-bf9a-d982f21c8aec None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:29.397754 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d8e285-21ca-4495-9f66-9bba6a60e31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:29.398389 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8e285-21ca-4495-9f66-9bba6a60e31b None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:29.398679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8e285-21ca-4495-9f66-9bba6a60e31b None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:29.398956 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8e285-21ca-4495-9f66-9bba6a60e31b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:29.477664 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:29.502848 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b678a624-885d-4ba7-807b-27da7feef111 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:29.504920 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:29.547434 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d8e285-21ca-4495-9f66-9bba6a60e31b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:29.548496 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fcb44a-8808-4b75-bdc4-015d1f6eb028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:29.549219 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d8e285-21ca-4495-9f66-9bba6a60e31b None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:29.549577 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fcb44a-8808-4b75-bdc4-015d1f6eb028 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:29.550005 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fcb44a-8808-4b75-bdc4-015d1f6eb028 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:29.550426 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fcb44a-8808-4b75-bdc4-015d1f6eb028 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:29.564315 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fcb44a-8808-4b75-bdc4-015d1f6eb028 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:29.572125 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-52fcb44a-8808-4b75-bdc4-015d1f6eb028 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 15:12:29.572125 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:29.572125 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:29.572125 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:29.572125 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:29.572125 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:29.572125 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:29.572125 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:29.572125 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:29.572125 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:29.572125 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:29.572125 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:29.572125 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:29.572125 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:29.572125 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:29.572125 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:29.576291 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:29.576291 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:29.576291 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:29.576291 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:29.576291 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:29.576291 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:29.576291 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:29.576291 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:29.576291 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:29.576291 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:29.576291 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:29.576291 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:29.576291 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:29.576291 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:29.576291 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:29.576291 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:29.577295 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:29.577295 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:29.577295 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:29.577295 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:29.577295 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:29.577295 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:29.577295 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:29.577295 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:29.577295 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:29.577295 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:29.577295 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 15:12:29.577295 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:29.578690 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 166/341] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 15:12:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:12:29.585269 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fcb44a-8808-4b75-bdc4-015d1f6eb028 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:29.725238 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fcb44a-8808-4b75-bdc4-015d1f6eb028 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:29.738210 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 176/342] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:12:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 15:12:29.738210 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8b0bd-b322-4c9d-95dd-3fd7352495d6 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:29.738210 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8b0bd-b322-4c9d-95dd-3fd7352495d6 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:29.738210 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8b0bd-b322-4c9d-95dd-3fd7352495d6 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:29.738210 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 177/343] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:29.743875 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f173fb-c8c0-4dce-91bb-a3527279ada4 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:29.744314 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f173fb-c8c0-4dce-91bb-a3527279ada4 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:29.744689 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f173fb-c8c0-4dce-91bb-a3527279ada4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:29.766095 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-12f173fb-c8c0-4dce-91bb-a3527279ada4 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:29.767315 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-12f173fb-c8c0-4dce-91bb-a3527279ada4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:29.790257 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e05f8-60b1-480f-9545-9cf74b33018a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:29.790895 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e05f8-60b1-480f-9545-9cf74b33018a None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:29.791179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e05f8-60b1-480f-9545-9cf74b33018a None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:29.791459 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e05f8-60b1-480f-9545-9cf74b33018a None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:29.792230 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79e05f8-60b1-480f-9545-9cf74b33018a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:29.792543 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79e05f8-60b1-480f-9545-9cf74b33018a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:29.794203 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79e05f8-60b1-480f-9545-9cf74b33018a None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:29.811436 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 167/344] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:12:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:12:29.812308 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e05f8-60b1-480f-9545-9cf74b33018a None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:29.820118 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:29.879548 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 178/345] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:29.881062 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f173fb-c8c0-4dce-91bb-a3527279ada4 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:29.900854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:29.958010 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e05f8-60b1-480f-9545-9cf74b33018a None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:30.001534 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8514a0-4db0-4bd8-a982-4908ef379c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:30.002112 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8514a0-4db0-4bd8-a982-4908ef379c7f None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:30.002399 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8514a0-4db0-4bd8-a982-4908ef379c7f None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:30.002680 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8514a0-4db0-4bd8-a982-4908ef379c7f None admin] PATH_INFO: `/v3/projects/b972ca1ff8ee4c558eeb45ec24303061` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:30.004158 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8514a0-4db0-4bd8-a982-4908ef379c7f None admin] RBAC: Authorizing `identity:get_project(project_id=b972ca1ff8ee4c558eeb45ec24303061)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:30.005771 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8514a0-4db0-4bd8-a982-4908ef379c7f None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:30.007782 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 168/346] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:12:29 2022] GET /identity/v3/projects/b972ca1ff8ee4c558eeb45ec24303061 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:30.009046 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c1788-730e-476a-9685-888bc664729f None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:30.009374 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c1788-730e-476a-9685-888bc664729f None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:30.009660 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c1788-730e-476a-9685-888bc664729f None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:30.011128 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 169/347] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:30.011836 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c1788-730e-476a-9685-888bc664729f None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:30.013606 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f173fb-c8c0-4dce-91bb-a3527279ada4 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:30.051621 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8df9c4-5c74-4f1b-bdbc-40bb320e30a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:30.052837 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8df9c4-5c74-4f1b-bdbc-40bb320e30a0 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:30.053122 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8df9c4-5c74-4f1b-bdbc-40bb320e30a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:30.053399 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8df9c4-5c74-4f1b-bdbc-40bb320e30a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:30.094781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:30.129138 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8df9c4-5c74-4f1b-bdbc-40bb320e30a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:30.130876 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8df9c4-5c74-4f1b-bdbc-40bb320e30a0 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:30.131046 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c1788-730e-476a-9685-888bc664729f None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:30.143210 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:30.169263 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27b4a1-a426-41fd-980b-a49a854dc070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:30.169540 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27b4a1-a426-41fd-980b-a49a854dc070 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:30.169917 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27b4a1-a426-41fd-980b-a49a854dc070 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:30.170355 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27b4a1-a426-41fd-980b-a49a854dc070 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:30.184015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf27b4a1-a426-41fd-980b-a49a854dc070 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:30.189881 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-cf27b4a1-a426-41fd-980b-a49a854dc070 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 15:12:30.189881 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:30.189881 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:30.189881 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:30.189881 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:30.189881 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:30.189881 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:30.189881 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:30.189881 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:30.189881 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:30.189881 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:30.189881 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:30.189881 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:30.189881 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:30.189881 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:30.189881 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:30.190872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:30.190872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:30.190872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:30.190872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:30.190872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:30.190872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:30.190872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:30.190872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:30.190872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:30.190872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:30.190872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:30.190872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:30.190872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:30.190872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:30.190872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:30.190872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:30.191893 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:30.191893 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:30.191893 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:30.191893 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:30.191893 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:30.191893 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:30.191893 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:30.191893 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:30.191893 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:30.191893 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:30.191893 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 15:12:30.191893 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:30.191893 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 170/348] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:12:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:30.196950 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38505a1-0e5b-43cc-8132-a69f86379afa None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:30.197397 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38505a1-0e5b-43cc-8132-a69f86379afa None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:30.197759 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38505a1-0e5b-43cc-8132-a69f86379afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:30.218368 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-d38505a1-0e5b-43cc-8132-a69f86379afa None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:30.219561 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-d38505a1-0e5b-43cc-8132-a69f86379afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:30.242851 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 179/349] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:12:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 15:12:30.243715 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8df9c4-5c74-4f1b-bdbc-40bb320e30a0 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:30.297797 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 171/350] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:30.298653 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38505a1-0e5b-43cc-8132-a69f86379afa None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:30.369660 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8df9c4-5c74-4f1b-bdbc-40bb320e30a0 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:30.381256 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:30.383183 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:30.416268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf3c93e-c28c-405f-97af-4f74c4f1270d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:30.416921 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf3c93e-c28c-405f-97af-4f74c4f1270d None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:30.417199 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf3c93e-c28c-405f-97af-4f74c4f1270d None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:30.417475 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf3c93e-c28c-405f-97af-4f74c4f1270d None admin] PATH_INFO: `/v3/groups/02515ae8bab14d9386d9b69742479fd2` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:30.426512 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf3c93e-c28c-405f-97af-4f74c4f1270d None admin] RBAC: Authorizing `identity:get_group(group_id=02515ae8bab14d9386d9b69742479fd2)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:30.428014 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf3c93e-c28c-405f-97af-4f74c4f1270d None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:30.428864 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38505a1-0e5b-43cc-8132-a69f86379afa None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:30.431370 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 180/351] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 5 15:12:30 2022] GET /identity/v3/groups/02515ae8bab14d9386d9b69742479fd2 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:30.431370 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf547376-aa9b-45a5-83a2-437f46e64748 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:30.431370 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf547376-aa9b-45a5-83a2-437f46e64748 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:30.433838 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf547376-aa9b-45a5-83a2-437f46e64748 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:30.433838 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 181/352] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:30.433838 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf547376-aa9b-45a5-83a2-437f46e64748 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:30.478942 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348f304-3e28-474c-b649-4eaca453f499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:30.479642 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348f304-3e28-474c-b649-4eaca453f499 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:30.479933 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348f304-3e28-474c-b649-4eaca453f499 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:30.480250 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348f304-3e28-474c-b649-4eaca453f499 None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:30.481024 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8348f304-3e28-474c-b649-4eaca453f499 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:30.481313 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8348f304-3e28-474c-b649-4eaca453f499 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:30.482688 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8348f304-3e28-474c-b649-4eaca453f499 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:30.494026 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 172/353] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 15:12:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:30.494978 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348f304-3e28-474c-b649-4eaca453f499 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:30.579607 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf547376-aa9b-45a5-83a2-437f46e64748 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:30.620240 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f534c5-91e6-462b-a2f3-af29d3f73691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:30.621021 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f534c5-91e6-462b-a2f3-af29d3f73691 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:30.621459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f534c5-91e6-462b-a2f3-af29d3f73691 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:30.621889 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f534c5-91e6-462b-a2f3-af29d3f73691 None admin] PATH_INFO: `/v3/projects/b972ca1ff8ee4c558eeb45ec24303061` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:30.624870 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f534c5-91e6-462b-a2f3-af29d3f73691 None admin] RBAC: Authorizing `identity:get_project(project_id=b972ca1ff8ee4c558eeb45ec24303061)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:30.626331 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f534c5-91e6-462b-a2f3-af29d3f73691 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:30.628099 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 182/354] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:12:30 2022] GET /identity/v3/projects/b972ca1ff8ee4c558eeb45ec24303061 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:30.629212 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a3680-d1ff-41dd-acca-b8d0ed85d2d6 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:30.629502 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a3680-d1ff-41dd-acca-b8d0ed85d2d6 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:30.629781 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a3680-d1ff-41dd-acca-b8d0ed85d2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:30.636784 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348f304-3e28-474c-b649-4eaca453f499 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:30.658314 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-497a3680-d1ff-41dd-acca-b8d0ed85d2d6 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:30.659224 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-497a3680-d1ff-41dd-acca-b8d0ed85d2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:30.669321 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:30.683603 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1ba752-0de4-49e0-a3a7-3eabfdc5bb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:30.684395 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ba752-0de4-49e0-a3a7-3eabfdc5bb83 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:30.684710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ba752-0de4-49e0-a3a7-3eabfdc5bb83 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:30.684968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ba752-0de4-49e0-a3a7-3eabfdc5bb83 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:30.702054 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1ba752-0de4-49e0-a3a7-3eabfdc5bb83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:30.704995 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:30.707602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-bb1ba752-0de4-49e0-a3a7-3eabfdc5bb83 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 15:12:30.707602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:30.707602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:30.707602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:30.707602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:30.707602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:30.707602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:30.707602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:30.707602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:30.707602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:30.707602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:30.707602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:30.707602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:30.707602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:30.707602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:30.707602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:30.709193 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:30.709193 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:30.709193 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:30.709193 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:30.709193 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:30.709193 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:30.709193 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:30.709193 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:30.709193 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:30.709193 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:30.709193 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:30.709193 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:30.709193 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:30.709193 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:30.709193 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:30.709193 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:30.709987 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:30.709987 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:30.709987 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:30.709987 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:30.709987 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:30.709987 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:30.709987 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:30.709987 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:30.709987 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:30.709987 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:30.709987 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 15:12:30.709987 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:30.709987 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 173/355] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:12:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:30.711692 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1ba752-0de4-49e0-a3a7-3eabfdc5bb83 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:30.749885 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 183/356] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:30.750731 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a3680-d1ff-41dd-acca-b8d0ed85d2d6 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:30.842992 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1ba752-0de4-49e0-a3a7-3eabfdc5bb83 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:30.877625 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a3680-d1ff-41dd-acca-b8d0ed85d2d6 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:30.888972 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f231a-70d9-4747-8200-8d0bce5b2075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:30.889819 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f231a-70d9-4747-8200-8d0bce5b2075 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:30.890254 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f231a-70d9-4747-8200-8d0bce5b2075 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:30.890531 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f231a-70d9-4747-8200-8d0bce5b2075 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:30.891321 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287f231a-70d9-4747-8200-8d0bce5b2075 None admin] RBAC: Adding query filter params (group.id=02515ae8bab14d9386d9b69742479fd2, role.id=d6c91838e4944cfbb23e1b6ecb009d26, scope.project.id=b972ca1ff8ee4c558eeb45ec24303061) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:30.891627 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287f231a-70d9-4747-8200-8d0bce5b2075 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:30.893124 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287f231a-70d9-4747-8200-8d0bce5b2075 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:30.901433 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 174/357] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Aug 5 15:12:30 2022] GET /identity/v3/role_assignments?group.id=02515ae8bab14d9386d9b69742479fd2&scope.project.id=b972ca1ff8ee4c558eeb45ec24303061&role.id=d6c91838e4944cfbb23e1b6ecb009d26 => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:30.901433 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f231a-70d9-4747-8200-8d0bce5b2075 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:30.924697 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a378b-2009-4bd3-97f6-464813c90760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:30.926484 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a378b-2009-4bd3-97f6-464813c90760 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:30.927029 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a378b-2009-4bd3-97f6-464813c90760 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:30.927786 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a378b-2009-4bd3-97f6-464813c90760 None admin] PATH_INFO: `/v3/users/b5c970ee793a40febb1fb633ab396773` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:30.930290 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9a378b-2009-4bd3-97f6-464813c90760 None admin] RBAC: Authorizing `identity:get_user(user_id=b5c970ee793a40febb1fb633ab396773)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:30.932781 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9a378b-2009-4bd3-97f6-464813c90760 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:30.935755 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 184/358] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:12:30 2022] GET /identity/v3/users/b5c970ee793a40febb1fb633ab396773 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:30.940317 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a378b-2009-4bd3-97f6-464813c90760 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:30.965270 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:30.990171 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:31.052789 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f231a-70d9-4747-8200-8d0bce5b2075 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:31.075312 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a378b-2009-4bd3-97f6-464813c90760 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:31.094906 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfafbd6d-c15b-4b74-b0db-bec43efc533c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:31.095497 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfafbd6d-c15b-4b74-b0db-bec43efc533c None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:31.095764 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfafbd6d-c15b-4b74-b0db-bec43efc533c None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:31.096040 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfafbd6d-c15b-4b74-b0db-bec43efc533c None admin] PATH_INFO: `/v3/groups/02515ae8bab14d9386d9b69742479fd2` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:31.097334 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfafbd6d-c15b-4b74-b0db-bec43efc533c None admin] RBAC: Authorizing `identity:get_group(group_id=02515ae8bab14d9386d9b69742479fd2)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:31.098944 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfafbd6d-c15b-4b74-b0db-bec43efc533c None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:31.100719 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 175/359] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 5 15:12:30 2022] GET /identity/v3/groups/02515ae8bab14d9386d9b69742479fd2 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:31.101469 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfafbd6d-c15b-4b74-b0db-bec43efc533c None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:31.122478 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60fd95-bf8d-450d-ae64-90a8ab3ddc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:31.123111 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60fd95-bf8d-450d-ae64-90a8ab3ddc65 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:31.123393 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60fd95-bf8d-450d-ae64-90a8ab3ddc65 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:31.123673 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60fd95-bf8d-450d-ae64-90a8ab3ddc65 None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:31.124697 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff60fd95-bf8d-450d-ae64-90a8ab3ddc65 None admin] RBAC: Adding query filter params (name=member) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:31.125003 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff60fd95-bf8d-450d-ae64-90a8ab3ddc65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:31.126376 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff60fd95-bf8d-450d-ae64-90a8ab3ddc65 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:31.143362 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 185/360] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 15:12:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:31.143362 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60fd95-bf8d-450d-ae64-90a8ab3ddc65 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:31.241526 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfafbd6d-c15b-4b74-b0db-bec43efc533c None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:31.268413 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60fd95-bf8d-450d-ae64-90a8ab3ddc65 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:31.268603 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:31.271261 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:31.293449 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dff397-734c-4c07-bd7a-0a0597ef64d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:31.294112 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dff397-734c-4c07-bd7a-0a0597ef64d0 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:31.294401 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dff397-734c-4c07-bd7a-0a0597ef64d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:31.294685 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dff397-734c-4c07-bd7a-0a0597ef64d0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:31.307300 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dff397-734c-4c07-bd7a-0a0597ef64d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:31.309755 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8103dffd-adf0-409d-adef-7bf50bc0ca17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:31.312211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103dffd-adf0-409d-adef-7bf50bc0ca17 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:31.312211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103dffd-adf0-409d-adef-7bf50bc0ca17 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:31.312211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103dffd-adf0-409d-adef-7bf50bc0ca17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:31.312211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8103dffd-adf0-409d-adef-7bf50bc0ca17 None admin] RBAC: Adding query filter params (role.id=09be81c3953242a08b57fd702a5082e3, user.id=b5c970ee793a40febb1fb633ab396773, scope.system=all) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:31.312211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8103dffd-adf0-409d-adef-7bf50bc0ca17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:31.313579 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8103dffd-adf0-409d-adef-7bf50bc0ca17 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:31.314093 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-43dff397-734c-4c07-bd7a-0a0597ef64d0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 15:12:31.314093 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:31.314093 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:31.314093 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:31.314093 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:31.314093 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:31.314093 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:31.314093 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:31.314093 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:31.314093 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:31.314093 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:31.314093 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:31.314093 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:31.314093 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:31.314093 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:31.314093 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:31.314944 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:31.314944 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:31.314944 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:31.314944 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:31.314944 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:31.314944 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:31.314944 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:31.314944 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:31.314944 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:31.314944 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:31.314944 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:31.314944 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:31.314944 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:31.314944 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:31.314944 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:31.314944 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:31.315736 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:31.315736 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:31.315736 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:31.315736 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:31.315736 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:31.315736 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:31.315736 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:31.315736 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:31.315736 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:31.315736 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:31.315736 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 15:12:31.315736 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:31.322565 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 176/361] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 15:12:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:31.322565 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45553e-49a4-46e1-82be-82526e909629 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:31.322565 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45553e-49a4-46e1-82be-82526e909629 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:31.322565 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45553e-49a4-46e1-82be-82526e909629 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:31.322565 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 177/362] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:31.322928 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc45553e-49a4-46e1-82be-82526e909629 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:31.336587 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 186/363] 10.222.0.40 () {56 vars in 1308 bytes} [Fri Aug 5 15:12:31 2022] GET /identity/v3/role_assignments?user.id=b5c970ee793a40febb1fb633ab396773&scope.system=all&role.id=09be81c3953242a08b57fd702a5082e3 => generated 510 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:31.338122 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8103dffd-adf0-409d-adef-7bf50bc0ca17 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:31.470890 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc45553e-49a4-46e1-82be-82526e909629 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:31.484761 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8103dffd-adf0-409d-adef-7bf50bc0ca17 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:31.522255 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac44cbcb-1b97-4101-aec5-9f4c4d9c3543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:31.523272 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac44cbcb-1b97-4101-aec5-9f4c4d9c3543 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:31.523556 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac44cbcb-1b97-4101-aec5-9f4c4d9c3543 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:31.524115 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac44cbcb-1b97-4101-aec5-9f4c4d9c3543 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:31.524923 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac44cbcb-1b97-4101-aec5-9f4c4d9c3543 None admin] RBAC: Adding query filter params (group.id=02515ae8bab14d9386d9b69742479fd2, role.id=944427ca14ee4c81a05ec67e54f50e7f, scope.project.id=b972ca1ff8ee4c558eeb45ec24303061) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:31.525274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac44cbcb-1b97-4101-aec5-9f4c4d9c3543 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:31.527095 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac44cbcb-1b97-4101-aec5-9f4c4d9c3543 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:31.528898 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5ed8fe-ca6a-4dd1-b3fd-05b090e82341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:31.529529 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ed8fe-ca6a-4dd1-b3fd-05b090e82341 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:31.529881 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ed8fe-ca6a-4dd1-b3fd-05b090e82341 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:31.530170 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ed8fe-ca6a-4dd1-b3fd-05b090e82341 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:31.533671 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 178/364] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Aug 5 15:12:31 2022] GET /identity/v3/role_assignments?group.id=02515ae8bab14d9386d9b69742479fd2&scope.project.id=b972ca1ff8ee4c558eeb45ec24303061&role.id=944427ca14ee4c81a05ec67e54f50e7f => generated 260 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:31.534351 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac44cbcb-1b97-4101-aec5-9f4c4d9c3543 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:31.571592 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:31.592873 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:31.632043 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5ed8fe-ca6a-4dd1-b3fd-05b090e82341 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:31.634516 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5ed8fe-ca6a-4dd1-b3fd-05b090e82341 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:31.678973 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac44cbcb-1b97-4101-aec5-9f4c4d9c3543 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:31.728475 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce2b353-e1da-4685-b9ba-71909e9168db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:31.729034 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2b353-e1da-4685-b9ba-71909e9168db None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:31.729364 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2b353-e1da-4685-b9ba-71909e9168db None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:31.729652 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2b353-e1da-4685-b9ba-71909e9168db None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:31.730387 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce2b353-e1da-4685-b9ba-71909e9168db None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:31.730753 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce2b353-e1da-4685-b9ba-71909e9168db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:31.733124 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce2b353-e1da-4685-b9ba-71909e9168db None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:31.743548 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 179/365] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:12:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:31.744929 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccaadaf-d8a4-4e20-ac7d-ca30276f7832 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:31.745222 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccaadaf-d8a4-4e20-ac7d-ca30276f7832 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:31.745500 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccaadaf-d8a4-4e20-ac7d-ca30276f7832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:31.767880 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-1ccaadaf-d8a4-4e20-ac7d-ca30276f7832 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:31.769136 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-1ccaadaf-d8a4-4e20-ac7d-ca30276f7832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:31.779836 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 187/366] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:12:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 15:12:31.780802 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5ed8fe-ca6a-4dd1-b3fd-05b090e82341 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:31.859317 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 180/367] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:31.860972 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccaadaf-d8a4-4e20-ac7d-ca30276f7832 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:31.878056 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:31.917786 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:31.939378 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5ed8fe-ca6a-4dd1-b3fd-05b090e82341 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:32.003263 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1609e8-4f41-492f-acdc-309991d4f480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:32.004325 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1609e8-4f41-492f-acdc-309991d4f480 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:32.004619 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1609e8-4f41-492f-acdc-309991d4f480 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:32.005221 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1609e8-4f41-492f-acdc-309991d4f480 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:32.025784 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccaadaf-d8a4-4e20-ac7d-ca30276f7832 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:32.070462 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6cab1-a856-468c-bbd7-d00345fb0adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:32.071208 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6cab1-a856-468c-bbd7-d00345fb0adc None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:32.071595 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6cab1-a856-468c-bbd7-d00345fb0adc None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:32.071882 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6cab1-a856-468c-bbd7-d00345fb0adc None admin] PATH_INFO: `/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:32.073106 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f6cab1-a856-468c-bbd7-d00345fb0adc None admin] RBAC: Authorizing `identity:get_project(project_id=759a514c6fa54b4b8fa90f2d4f181f99)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:32.074804 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f6cab1-a856-468c-bbd7-d00345fb0adc None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:32.076604 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 181/368] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:12:31 2022] GET /identity/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:32.077383 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6cab1-a856-468c-bbd7-d00345fb0adc None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:32.085127 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1609e8-4f41-492f-acdc-309991d4f480 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:32.086607 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1609e8-4f41-492f-acdc-309991d4f480 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:32.147770 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:32.197446 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6cab1-a856-468c-bbd7-d00345fb0adc None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:32.204599 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 188/369] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:12:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 15:12:32.205483 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1609e8-4f41-492f-acdc-309991d4f480 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:32.208875 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:32.237730 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347fc8eb-1aa9-4e01-b76f-bfe8c254518e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:32.238674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347fc8eb-1aa9-4e01-b76f-bfe8c254518e None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:32.238967 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347fc8eb-1aa9-4e01-b76f-bfe8c254518e None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:32.239797 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347fc8eb-1aa9-4e01-b76f-bfe8c254518e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:32.256983 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347fc8eb-1aa9-4e01-b76f-bfe8c254518e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:32.265740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-347fc8eb-1aa9-4e01-b76f-bfe8c254518e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:12:32.265740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:32.265740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:32.265740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:32.265740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:32.265740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:32.265740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:32.265740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:32.265740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:32.265740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:32.265740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:32.265740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:32.265740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:32.265740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:32.265740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:32.265740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:32.266676 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:32.266676 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:32.266676 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:32.266676 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:32.266676 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:32.266676 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:32.266676 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:32.266676 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:32.266676 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:32.266676 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:32.266676 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:32.266676 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:32.266676 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:32.266676 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:32.266676 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:32.266676 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:32.267468 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:32.267468 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:32.267468 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:32.267468 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:32.267468 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:32.267468 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:32.267468 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:32.267468 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:32.267468 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:32.267468 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:32.267468 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:12:32.267468 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:32.267468 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 182/370] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 15:12:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:32.268277 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347fc8eb-1aa9-4e01-b76f-bfe8c254518e None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:32.337639 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1609e8-4f41-492f-acdc-309991d4f480 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:32.382691 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb45ce-2207-4996-acbe-229c88499d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:32.384176 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb45ce-2207-4996-acbe-229c88499d6b None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:32.384672 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb45ce-2207-4996-acbe-229c88499d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:32.385401 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb45ce-2207-4996-acbe-229c88499d6b None admin] PATH_INFO: `/v3/groups/02515ae8bab14d9386d9b69742479fd2` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:32.387239 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfb45ce-2207-4996-acbe-229c88499d6b None admin] RBAC: Authorizing `identity:get_group(group_id=02515ae8bab14d9386d9b69742479fd2)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:32.388932 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfb45ce-2207-4996-acbe-229c88499d6b None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:32.391333 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347fc8eb-1aa9-4e01-b76f-bfe8c254518e None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:32.391583 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 189/371] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 5 15:12:32 2022] GET /identity/v3/groups/02515ae8bab14d9386d9b69742479fd2 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:32.392653 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb45ce-2207-4996-acbe-229c88499d6b None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:32.429787 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:32.433076 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2bc1a6-f451-4f16-94a8-3f2a8efbb545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:32.433497 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2bc1a6-f451-4f16-94a8-3f2a8efbb545 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:32.433959 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2bc1a6-f451-4f16-94a8-3f2a8efbb545 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:32.434263 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2bc1a6-f451-4f16-94a8-3f2a8efbb545 None admin] PATH_INFO: `/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:32.435473 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2bc1a6-f451-4f16-94a8-3f2a8efbb545 None admin] RBAC: Authorizing `identity:get_project(project_id=759a514c6fa54b4b8fa90f2d4f181f99)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:32.437017 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2bc1a6-f451-4f16-94a8-3f2a8efbb545 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:32.439733 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 183/372] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:12:32 2022] GET /identity/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:32.440628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2bc1a6-f451-4f16-94a8-3f2a8efbb545 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:32.474920 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:32.522772 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb45ce-2207-4996-acbe-229c88499d6b None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:32.567912 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee9b65-acc2-4662-9e0c-4e122bfe46eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:32.568946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee9b65-acc2-4662-9e0c-4e122bfe46eb None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:32.569434 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee9b65-acc2-4662-9e0c-4e122bfe46eb None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:32.569911 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee9b65-acc2-4662-9e0c-4e122bfe46eb None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:32.570826 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ee9b65-acc2-4662-9e0c-4e122bfe46eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:32.571351 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ee9b65-acc2-4662-9e0c-4e122bfe46eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:32.572963 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ee9b65-acc2-4662-9e0c-4e122bfe46eb None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:32.573805 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2bc1a6-f451-4f16-94a8-3f2a8efbb545 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:32.583360 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 190/373] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:12:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:32.584523 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee9b65-acc2-4662-9e0c-4e122bfe46eb None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:32.609172 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956acddf-a47b-4101-b9c0-6fe275f10d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:32.609795 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956acddf-a47b-4101-b9c0-6fe275f10d30 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:32.610079 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956acddf-a47b-4101-b9c0-6fe275f10d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:32.610356 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956acddf-a47b-4101-b9c0-6fe275f10d30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:32.611149 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956acddf-a47b-4101-b9c0-6fe275f10d30 None admin] RBAC: Adding query filter params (group.id=02515ae8bab14d9386d9b69742479fd2, role.id=d6c91838e4944cfbb23e1b6ecb009d26, scope.project.id=759a514c6fa54b4b8fa90f2d4f181f99) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:32.611463 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956acddf-a47b-4101-b9c0-6fe275f10d30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:32.614096 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956acddf-a47b-4101-b9c0-6fe275f10d30 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:32.624476 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 184/374] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Aug 5 15:12:32 2022] GET /identity/v3/role_assignments?group.id=02515ae8bab14d9386d9b69742479fd2&scope.project.id=759a514c6fa54b4b8fa90f2d4f181f99&role.id=d6c91838e4944cfbb23e1b6ecb009d26 => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:32.625420 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956acddf-a47b-4101-b9c0-6fe275f10d30 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:32.686716 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:32.703238 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee9b65-acc2-4662-9e0c-4e122bfe46eb None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:32.752094 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:32.758651 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a8dae-13f9-414a-910a-e8fa11147677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:32.759575 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a8dae-13f9-414a-910a-e8fa11147677 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:32.760059 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a8dae-13f9-414a-910a-e8fa11147677 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:32.760548 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a8dae-13f9-414a-910a-e8fa11147677 None admin] PATH_INFO: `/v3/groups/02515ae8bab14d9386d9b69742479fd2` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:32.764873 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956acddf-a47b-4101-b9c0-6fe275f10d30 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:32.765023 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945a8dae-13f9-414a-910a-e8fa11147677 None admin] RBAC: Authorizing `identity:get_group(group_id=02515ae8bab14d9386d9b69742479fd2)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:32.766606 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945a8dae-13f9-414a-910a-e8fa11147677 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:32.768836 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 191/375] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 5 15:12:32 2022] GET /identity/v3/groups/02515ae8bab14d9386d9b69742479fd2 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:32.774653 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a8dae-13f9-414a-910a-e8fa11147677 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:32.797313 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcea7c0-e65a-4ec5-bf8e-7508479e2b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:32.797951 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcea7c0-e65a-4ec5-bf8e-7508479e2b28 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:32.798242 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcea7c0-e65a-4ec5-bf8e-7508479e2b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:32.798528 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcea7c0-e65a-4ec5-bf8e-7508479e2b28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:32.876391 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcea7c0-e65a-4ec5-bf8e-7508479e2b28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:32.877866 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcea7c0-e65a-4ec5-bf8e-7508479e2b28 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:32.904504 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a8dae-13f9-414a-910a-e8fa11147677 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:32.944745 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e74a3d-0b22-4876-b518-8c3f2bd795a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:32.945673 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e74a3d-0b22-4876-b518-8c3f2bd795a0 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:32.946138 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e74a3d-0b22-4876-b518-8c3f2bd795a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:32.946624 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e74a3d-0b22-4876-b518-8c3f2bd795a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:32.947600 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e74a3d-0b22-4876-b518-8c3f2bd795a0 None admin] RBAC: Adding query filter params (group.id=02515ae8bab14d9386d9b69742479fd2, role.id=944427ca14ee4c81a05ec67e54f50e7f, scope.project.id=759a514c6fa54b4b8fa90f2d4f181f99) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:32.948121 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e74a3d-0b22-4876-b518-8c3f2bd795a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:32.949758 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e74a3d-0b22-4876-b518-8c3f2bd795a0 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:32.956624 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 192/376] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Aug 5 15:12:32 2022] GET /identity/v3/role_assignments?group.id=02515ae8bab14d9386d9b69742479fd2&scope.project.id=759a514c6fa54b4b8fa90f2d4f181f99&role.id=944427ca14ee4c81a05ec67e54f50e7f => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:32.978744 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:32.988782 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 185/377] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:12:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 15:12:32.995523 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e74a3d-0b22-4876-b518-8c3f2bd795a0 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:33.030458 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:33.118503 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e74a3d-0b22-4876-b518-8c3f2bd795a0 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:33.155702 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82671352-bfde-4bde-8060-8639ce1f4d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:33.156406 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82671352-bfde-4bde-8060-8639ce1f4d86 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:33.156696 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82671352-bfde-4bde-8060-8639ce1f4d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:33.156981 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82671352-bfde-4bde-8060-8639ce1f4d86 None admin] PATH_INFO: `/v3/projects/0744d3ce240a4f198b94f2d1e6d62c8e` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:33.158204 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82671352-bfde-4bde-8060-8639ce1f4d86 None admin] RBAC: Authorizing `identity:get_project(project_id=0744d3ce240a4f198b94f2d1e6d62c8e)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:33.159647 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82671352-bfde-4bde-8060-8639ce1f4d86 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:33.162235 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 193/378] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:12:32 2022] GET /identity/v3/projects/0744d3ce240a4f198b94f2d1e6d62c8e => generated 344 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:33.178485 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcea7c0-e65a-4ec5-bf8e-7508479e2b28 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:33.194382 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4348c-b19a-4415-a06b-54cf124fe81d None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:33.194684 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4348c-b19a-4415-a06b-54cf124fe81d None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:33.194961 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4348c-b19a-4415-a06b-54cf124fe81d None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:33.196645 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 194/379] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 15:12:33.202421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f5cb6c-321c-4f12-8c7a-016d9836fc27 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:33.202717 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f5cb6c-321c-4f12-8c7a-016d9836fc27 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:33.202995 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f5cb6c-321c-4f12-8c7a-016d9836fc27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:33.225860 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-49f5cb6c-321c-4f12-8c7a-016d9836fc27 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:33.227084 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-49f5cb6c-321c-4f12-8c7a-016d9836fc27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:33.312227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 195/380] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 15:12:33.320029 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcea7c0-e65a-4ec5-bf8e-7508479e2b28 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:33.350098 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f5cb6c-321c-4f12-8c7a-016d9836fc27 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:33.379044 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:33.388250 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aada9ab3-d10b-4277-856e-b0f40c0a3a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:33.388919 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada9ab3-d10b-4277-856e-b0f40c0a3a69 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:33.389210 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada9ab3-d10b-4277-856e-b0f40c0a3a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:33.389495 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada9ab3-d10b-4277-856e-b0f40c0a3a69 None admin] PATH_INFO: `/v3/groups/d178fe555c6d4070adcb0068db625fa9` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:33.402624 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aada9ab3-d10b-4277-856e-b0f40c0a3a69 None admin] RBAC: Authorizing `identity:get_group(group_id=d178fe555c6d4070adcb0068db625fa9)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:33.404119 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aada9ab3-d10b-4277-856e-b0f40c0a3a69 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:33.405826 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 186/381] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 5 15:12:33 2022] GET /identity/v3/groups/d178fe555c6d4070adcb0068db625fa9 => generated 229 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:33.412572 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aada9ab3-d10b-4277-856e-b0f40c0a3a69 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:33.498941 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f5cb6c-321c-4f12-8c7a-016d9836fc27 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:33.507048 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:33.555300 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba9fd6b-257d-45d8-ad17-d9a74894b1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:33.555961 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9fd6b-257d-45d8-ad17-d9a74894b1a5 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:33.556281 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9fd6b-257d-45d8-ad17-d9a74894b1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:33.556560 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9fd6b-257d-45d8-ad17-d9a74894b1a5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:33.571178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba9fd6b-257d-45d8-ad17-d9a74894b1a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:33.577072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-7ba9fd6b-257d-45d8-ad17-d9a74894b1a5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 15:12:33.577072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:33.577072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:33.577072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:33.577072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:33.577072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:33.577072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:33.577072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:33.577072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:33.577072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:33.577072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:33.577072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:33.577072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:33.577072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:33.577072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:33.577072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:33.577925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:33.577925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:33.577925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:33.577925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:33.577925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:33.577925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:33.577925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:33.577925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:33.577925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:33.577925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:33.577925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:33.577925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:33.577925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:33.577925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:33.577925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:33.577925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:33.578717 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:33.578717 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:33.578717 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:33.578717 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:33.578717 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:33.578717 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:33.578717 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:33.578717 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:33.578717 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:33.578717 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:33.578717 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 15:12:33.578717 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:33.579321 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aada9ab3-d10b-4277-856e-b0f40c0a3a69 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:33.579392 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 196/382] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:12:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:12:33.586308 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba9fd6b-257d-45d8-ad17-d9a74894b1a5 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:33.629110 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dec511-fda0-493c-88a2-96a37b90d561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:33.629769 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dec511-fda0-493c-88a2-96a37b90d561 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:33.630057 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dec511-fda0-493c-88a2-96a37b90d561 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:33.630336 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dec511-fda0-493c-88a2-96a37b90d561 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:33.631183 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dec511-fda0-493c-88a2-96a37b90d561 None admin] RBAC: Adding query filter params (group.id=d178fe555c6d4070adcb0068db625fa9, role.id=7c88c815db854954b516ab97cd471647, scope.project.id=0744d3ce240a4f198b94f2d1e6d62c8e) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:33.631542 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dec511-fda0-493c-88a2-96a37b90d561 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:33.633130 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dec511-fda0-493c-88a2-96a37b90d561 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:33.642017 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 187/383] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Aug 5 15:12:33 2022] GET /identity/v3/role_assignments?group.id=d178fe555c6d4070adcb0068db625fa9&scope.project.id=0744d3ce240a4f198b94f2d1e6d62c8e&role.id=7c88c815db854954b516ab97cd471647 => generated 260 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:12:33.729099 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:33.755408 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba9fd6b-257d-45d8-ad17-d9a74894b1a5 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:33.805968 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b6aef-1918-478a-93fa-dc962cd59df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:33.806601 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b6aef-1918-478a-93fa-dc962cd59df8 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:33.806883 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b6aef-1918-478a-93fa-dc962cd59df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:33.807222 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b6aef-1918-478a-93fa-dc962cd59df8 None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:33.807952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5b6aef-1918-478a-93fa-dc962cd59df8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:33.808338 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5b6aef-1918-478a-93fa-dc962cd59df8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:33.809726 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5b6aef-1918-478a-93fa-dc962cd59df8 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:33.819603 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 197/384] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 15:12:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:12:33.829329 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dec511-fda0-493c-88a2-96a37b90d561 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:33.963610 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dec511-fda0-493c-88a2-96a37b90d561 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:33.989489 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:34.005232 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46fef67-98ce-4cd1-a150-7db3fb907516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:34.005897 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46fef67-98ce-4cd1-a150-7db3fb907516 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:34.006183 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46fef67-98ce-4cd1-a150-7db3fb907516 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:34.006470 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46fef67-98ce-4cd1-a150-7db3fb907516 None admin] PATH_INFO: `/v3/groups/02515ae8bab14d9386d9b69742479fd2` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:34.010268 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46fef67-98ce-4cd1-a150-7db3fb907516 None admin] RBAC: Authorizing `identity:get_group(group_id=02515ae8bab14d9386d9b69742479fd2)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:34.010268 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46fef67-98ce-4cd1-a150-7db3fb907516 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:34.011329 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 188/385] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 5 15:12:33 2022] GET /identity/v3/groups/02515ae8bab14d9386d9b69742479fd2 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:34.017335 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b6aef-1918-478a-93fa-dc962cd59df8 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:34.106858 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8c925-d09b-491b-af2e-5e0f1e2fe241 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:34.107183 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8c925-d09b-491b-af2e-5e0f1e2fe241 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:34.107345 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8c925-d09b-491b-af2e-5e0f1e2fe241 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:34.108955 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 189/386] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:34.115312 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c853b35-8256-490f-b41b-b2edf9922b99 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:34.115593 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c853b35-8256-490f-b41b-b2edf9922b99 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:34.115866 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c853b35-8256-490f-b41b-b2edf9922b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:34.141442 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b6aef-1918-478a-93fa-dc962cd59df8 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:34.157104 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-0c853b35-8256-490f-b41b-b2edf9922b99 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:34.158316 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-0c853b35-8256-490f-b41b-b2edf9922b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:34.226739 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa37b1e3-df64-462e-8d65-444535c810f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:34.227387 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37b1e3-df64-462e-8d65-444535c810f9 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:34.227670 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37b1e3-df64-462e-8d65-444535c810f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:34.227946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37b1e3-df64-462e-8d65-444535c810f9 None admin] PATH_INFO: `/v3/projects/b972ca1ff8ee4c558eeb45ec24303061` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:34.229932 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa37b1e3-df64-462e-8d65-444535c810f9 None admin] RBAC: Authorizing `identity:get_project(project_id=b972ca1ff8ee4c558eeb45ec24303061)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:34.231375 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa37b1e3-df64-462e-8d65-444535c810f9 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:34.234130 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 198/387] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:12:34 2022] GET /identity/v3/projects/b972ca1ff8ee4c558eeb45ec24303061 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:34.242502 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa37b1e3-df64-462e-8d65-444535c810f9 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:34.267079 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:34.279071 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 190/388] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 15:12:34.320619 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c853b35-8256-490f-b41b-b2edf9922b99 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:34.426436 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa37b1e3-df64-462e-8d65-444535c810f9 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:34.490945 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78033368-68a9-4834-8e5a-dd7c3dd141f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:34.491627 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78033368-68a9-4834-8e5a-dd7c3dd141f7 None admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:34.491912 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78033368-68a9-4834-8e5a-dd7c3dd141f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:34.492219 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78033368-68a9-4834-8e5a-dd7c3dd141f7 None admin] PATH_INFO: `/v3/projects/b972ca1ff8ee4c558eeb45ec24303061/groups/02515ae8bab14d9386d9b69742479fd2/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:34.494918 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78033368-68a9-4834-8e5a-dd7c3dd141f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=b972ca1ff8ee4c558eeb45ec24303061, group_id=02515ae8bab14d9386d9b69742479fd2, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:34.497531 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78033368-68a9-4834-8e5a-dd7c3dd141f7 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:34.498568 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c853b35-8256-490f-b41b-b2edf9922b99 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:34.524993 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:34.547461 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba14d55e-d5a1-4db9-ab6c-ff5c8743245b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:34.548438 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14d55e-d5a1-4db9-ab6c-ff5c8743245b None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:34.548980 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14d55e-d5a1-4db9-ab6c-ff5c8743245b None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:34.549404 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14d55e-d5a1-4db9-ab6c-ff5c8743245b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:34.563299 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba14d55e-d5a1-4db9-ab6c-ff5c8743245b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:34.569499 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-ba14d55e-d5a1-4db9-ab6c-ff5c8743245b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 15:12:34.569499 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:34.569499 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:34.569499 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:34.569499 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:34.569499 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:34.569499 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:34.569499 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:34.569499 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:34.569499 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:34.569499 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:34.569499 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:34.569499 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:34.569499 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:34.569499 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:34.569499 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:34.570581 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:34.570581 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:34.570581 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:34.570581 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:34.570581 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:34.570581 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:34.570581 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:34.570581 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:34.570581 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:34.570581 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:34.570581 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:34.570581 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:34.570581 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:34.570581 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:34.570581 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:34.570581 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:34.571412 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:34.571412 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:34.571412 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:34.571412 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:34.571412 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:34.571412 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:34.571412 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:34.571412 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:34.571412 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:34.571412 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:34.571412 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 15:12:34.571412 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:34.571412 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 191/389] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 15:12:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:12:34.577119 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba14d55e-d5a1-4db9-ab6c-ff5c8743245b None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:34.593976 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 199/390] 10.222.0.40 () {58 vars in 1442 bytes} [Fri Aug 5 15:12:34 2022] PUT /identity/v3/projects/b972ca1ff8ee4c558eeb45ec24303061/groups/02515ae8bab14d9386d9b69742479fd2/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:12:34.699473 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba14d55e-d5a1-4db9-ab6c-ff5c8743245b None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:34.737502 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1595736c-6f2f-490f-83b8-92144ecc9690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:34.738510 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1595736c-6f2f-490f-83b8-92144ecc9690 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:34.739056 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1595736c-6f2f-490f-83b8-92144ecc9690 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:34.740035 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1595736c-6f2f-490f-83b8-92144ecc9690 None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:34.741032 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1595736c-6f2f-490f-83b8-92144ecc9690 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:34.741554 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1595736c-6f2f-490f-83b8-92144ecc9690 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:34.743208 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1595736c-6f2f-490f-83b8-92144ecc9690 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:34.753424 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 192/391] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:12:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:34.760488 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78033368-68a9-4834-8e5a-dd7c3dd141f7 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:34.841127 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:34.872961 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78033368-68a9-4834-8e5a-dd7c3dd141f7 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:34.909715 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2fede4-8ae8-40bc-84da-093084b133a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:34.910777 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2fede4-8ae8-40bc-84da-093084b133a9 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:34.911827 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2fede4-8ae8-40bc-84da-093084b133a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:34.912354 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2fede4-8ae8-40bc-84da-093084b133a9 None admin] PATH_INFO: `/v3/groups/02515ae8bab14d9386d9b69742479fd2` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:34.913852 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2fede4-8ae8-40bc-84da-093084b133a9 None admin] RBAC: Authorizing `identity:get_group(group_id=02515ae8bab14d9386d9b69742479fd2)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:34.915564 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2fede4-8ae8-40bc-84da-093084b133a9 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:34.917739 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 200/392] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 5 15:12:34 2022] GET /identity/v3/groups/02515ae8bab14d9386d9b69742479fd2 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:34.924156 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1595736c-6f2f-490f-83b8-92144ecc9690 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:34.969855 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:35.033166 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1595736c-6f2f-490f-83b8-92144ecc9690 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:35.069093 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d079c-0b3f-42f1-857d-c9313edc4293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:35.070116 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d079c-0b3f-42f1-857d-c9313edc4293 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:35.070619 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d079c-0b3f-42f1-857d-c9313edc4293 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:35.071092 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d079c-0b3f-42f1-857d-c9313edc4293 None admin] PATH_INFO: `/v3/projects/b972ca1ff8ee4c558eeb45ec24303061` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:35.072571 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d079c-0b3f-42f1-857d-c9313edc4293 None admin] RBAC: Authorizing `identity:get_project(project_id=b972ca1ff8ee4c558eeb45ec24303061)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:35.074220 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d079c-0b3f-42f1-857d-c9313edc4293 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:35.076493 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 193/393] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:12:34 2022] GET /identity/v3/projects/b972ca1ff8ee4c558eeb45ec24303061 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:35.084498 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2fede4-8ae8-40bc-84da-093084b133a9 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:35.209987 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2fede4-8ae8-40bc-84da-093084b133a9 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:35.246570 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c4dfbc-0e1a-4a64-b11f-b9ea0340872b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:35.247739 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4dfbc-0e1a-4a64-b11f-b9ea0340872b None admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:35.248295 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4dfbc-0e1a-4a64-b11f-b9ea0340872b None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:35.248775 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4dfbc-0e1a-4a64-b11f-b9ea0340872b None admin] PATH_INFO: `/v3/projects/b972ca1ff8ee4c558eeb45ec24303061/groups/02515ae8bab14d9386d9b69742479fd2/roles/944427ca14ee4c81a05ec67e54f50e7f` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:35.251425 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c4dfbc-0e1a-4a64-b11f-b9ea0340872b None admin] RBAC: Authorizing `identity:create_grant(project_id=b972ca1ff8ee4c558eeb45ec24303061, group_id=02515ae8bab14d9386d9b69742479fd2, role_id=944427ca14ee4c81a05ec67e54f50e7f)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:35.253137 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c4dfbc-0e1a-4a64-b11f-b9ea0340872b None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:35.326726 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:35.333950 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 201/394] 10.222.0.40 () {58 vars in 1442 bytes} [Fri Aug 5 15:12:35 2022] PUT /identity/v3/projects/b972ca1ff8ee4c558eeb45ec24303061/groups/02515ae8bab14d9386d9b69742479fd2/roles/944427ca14ee4c81a05ec67e54f50e7f => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:12:35.589759 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0873583-59a8-4f87-b78c-277e680eac9b None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:35.590361 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0873583-59a8-4f87-b78c-277e680eac9b None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:35.590818 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0873583-59a8-4f87-b78c-277e680eac9b None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:35.592206 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 194/395] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:35.597670 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde275e1-81b2-4097-8e58-fdce56890370 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:35.597973 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde275e1-81b2-4097-8e58-fdce56890370 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:35.598282 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde275e1-81b2-4097-8e58-fdce56890370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:35.617274 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-dde275e1-81b2-4097-8e58-fdce56890370 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:35.618539 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-dde275e1-81b2-4097-8e58-fdce56890370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:35.652218 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3137481d-25cc-44c0-98de-bfb536d88e50 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:35.652512 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3137481d-25cc-44c0-98de-bfb536d88e50 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:35.652775 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3137481d-25cc-44c0-98de-bfb536d88e50 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:35.654142 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 195/396] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 15:12:35.660967 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9993e0-60ef-44d8-bdf8-7698e4428971 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:35.661643 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9993e0-60ef-44d8-bdf8-7698e4428971 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:35.662963 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9993e0-60ef-44d8-bdf8-7698e4428971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:35.698258 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-db9993e0-60ef-44d8-bdf8-7698e4428971 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:35.701385 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-db9993e0-60ef-44d8-bdf8-7698e4428971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:35.702535 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 202/397] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:35.752677 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde275e1-81b2-4097-8e58-fdce56890370 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:35.829081 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 196/398] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 15:12:35.875967 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9993e0-60ef-44d8-bdf8-7698e4428971 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:35.921594 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde275e1-81b2-4097-8e58-fdce56890370 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:35.939771 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:35.960556 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:35.968434 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060633d8-8df6-4730-8bcd-d76f2204ebd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:35.969089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060633d8-8df6-4730-8bcd-d76f2204ebd4 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:35.969376 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060633d8-8df6-4730-8bcd-d76f2204ebd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:35.969654 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060633d8-8df6-4730-8bcd-d76f2204ebd4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:35.980718 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060633d8-8df6-4730-8bcd-d76f2204ebd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:35.985963 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-060633d8-8df6-4730-8bcd-d76f2204ebd4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 15:12:35.985963 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:35.985963 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:35.985963 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:35.985963 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:35.985963 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:35.985963 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:35.985963 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:35.985963 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:35.985963 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:35.985963 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:35.985963 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:35.985963 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:35.985963 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:35.985963 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:35.985963 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:35.986830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:35.986830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:35.986830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:35.986830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:35.986830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:35.986830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:35.986830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:35.986830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:35.986830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:35.986830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:35.986830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:35.986830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:35.986830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:35.986830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:35.986830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:35.986830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:35.987896 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:35.987896 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:35.987896 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:35.987896 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:35.987896 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:35.987896 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:35.987896 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:35.987896 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:35.987896 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:35.987896 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:35.987896 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 15:12:35.987896 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:35.987896 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 203/399] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:12:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:12:36.001027 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060633d8-8df6-4730-8bcd-d76f2204ebd4 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:36.006816 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9993e0-60ef-44d8-bdf8-7698e4428971 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:36.046016 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091c5f0-18e1-44ab-a517-e2a6a358f17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:36.046677 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091c5f0-18e1-44ab-a517-e2a6a358f17d None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:36.046963 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091c5f0-18e1-44ab-a517-e2a6a358f17d None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:36.047240 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091c5f0-18e1-44ab-a517-e2a6a358f17d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:36.058443 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c091c5f0-18e1-44ab-a517-e2a6a358f17d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:36.063986 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-c091c5f0-18e1-44ab-a517-e2a6a358f17d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 15:12:36.063986 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:36.063986 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:36.063986 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:36.063986 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:36.063986 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:36.063986 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:36.063986 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:36.063986 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:36.063986 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:36.063986 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:36.063986 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:36.063986 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:36.063986 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:36.063986 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:36.063986 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:36.064905 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:36.064905 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:36.064905 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:36.064905 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:36.064905 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:36.064905 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:36.064905 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:36.064905 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:36.064905 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:36.064905 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:36.064905 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:36.064905 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:36.064905 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:36.064905 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:36.064905 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:36.064905 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:36.066132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:36.066132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:36.066132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:36.066132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:36.066132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:36.066132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:36.066132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:36.066132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:36.066132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:36.066132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:36.066132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 15:12:36.066132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:36.066132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 197/400] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 15:12:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:36.075525 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091c5f0-18e1-44ab-a517-e2a6a358f17d None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:36.137416 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060633d8-8df6-4730-8bcd-d76f2204ebd4 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:36.174361 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db5c06c-efd9-4a84-a3d0-be5311a8d6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:36.174986 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5c06c-efd9-4a84-a3d0-be5311a8d6eb None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:36.175270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5c06c-efd9-4a84-a3d0-be5311a8d6eb None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:36.180737 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5c06c-efd9-4a84-a3d0-be5311a8d6eb None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:36.180737 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db5c06c-efd9-4a84-a3d0-be5311a8d6eb None admin] RBAC: Adding query filter params (name=member) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:36.180737 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db5c06c-efd9-4a84-a3d0-be5311a8d6eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:36.180737 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db5c06c-efd9-4a84-a3d0-be5311a8d6eb None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:36.187247 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 204/401] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 15:12:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:36.193328 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db5c06c-efd9-4a84-a3d0-be5311a8d6eb None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:36.266009 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:36.271729 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:36.278264 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091c5f0-18e1-44ab-a517-e2a6a358f17d None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:36.336636 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a91c4-24fb-4bee-bf86-cfd1278a9539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:36.337278 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a91c4-24fb-4bee-bf86-cfd1278a9539 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:36.337830 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a91c4-24fb-4bee-bf86-cfd1278a9539 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:36.338440 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a91c4-24fb-4bee-bf86-cfd1278a9539 None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:36.339332 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2a91c4-24fb-4bee-bf86-cfd1278a9539 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:36.339656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2a91c4-24fb-4bee-bf86-cfd1278a9539 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:36.341072 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2a91c4-24fb-4bee-bf86-cfd1278a9539 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:36.341667 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db5c06c-efd9-4a84-a3d0-be5311a8d6eb None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:36.350078 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 198/402] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:12:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:36.351482 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0163ce-cb02-4220-a32a-ae6a66e659e2 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:36.351777 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0163ce-cb02-4220-a32a-ae6a66e659e2 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:36.352058 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0163ce-cb02-4220-a32a-ae6a66e659e2 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:36.357492 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 199/403] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:36.358277 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0163ce-cb02-4220-a32a-ae6a66e659e2 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:36.377953 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18836c4-c90a-4438-bef6-dd23b9cb2e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:36.378620 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18836c4-c90a-4438-bef6-dd23b9cb2e11 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:36.378980 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18836c4-c90a-4438-bef6-dd23b9cb2e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:36.379263 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18836c4-c90a-4438-bef6-dd23b9cb2e11 None admin] PATH_INFO: `/v3/groups/02515ae8bab14d9386d9b69742479fd2` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:36.381057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18836c4-c90a-4438-bef6-dd23b9cb2e11 None admin] RBAC: Authorizing `identity:get_group(group_id=02515ae8bab14d9386d9b69742479fd2)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:36.382490 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18836c4-c90a-4438-bef6-dd23b9cb2e11 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:36.384506 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 205/404] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 5 15:12:36 2022] GET /identity/v3/groups/02515ae8bab14d9386d9b69742479fd2 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:36.390256 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af5b8f-3f8d-473f-943d-28bf993ff994 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:36.390480 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af5b8f-3f8d-473f-943d-28bf993ff994 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:36.390769 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af5b8f-3f8d-473f-943d-28bf993ff994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:36.409549 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-b5af5b8f-3f8d-473f-943d-28bf993ff994 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:36.410906 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-b5af5b8f-3f8d-473f-943d-28bf993ff994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:36.468665 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0163ce-cb02-4220-a32a-ae6a66e659e2 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:36.488091 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 206/405] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:36.488091 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af5b8f-3f8d-473f-943d-28bf993ff994 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:36.524154 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0098e0-4f0b-41da-9054-cf6713a5e3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:36.525189 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0098e0-4f0b-41da-9054-cf6713a5e3e4 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:36.525657 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0098e0-4f0b-41da-9054-cf6713a5e3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:36.526172 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0098e0-4f0b-41da-9054-cf6713a5e3e4 None admin] PATH_INFO: `/v3/groups/02515ae8bab14d9386d9b69742479fd2` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:36.527579 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0098e0-4f0b-41da-9054-cf6713a5e3e4 None admin] RBAC: Authorizing `identity:get_group(group_id=02515ae8bab14d9386d9b69742479fd2)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:36.532337 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0098e0-4f0b-41da-9054-cf6713a5e3e4 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:36.533623 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:36.535448 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 200/406] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 5 15:12:36 2022] GET /identity/v3/groups/02515ae8bab14d9386d9b69742479fd2 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:36.536481 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0098e0-4f0b-41da-9054-cf6713a5e3e4 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:36.570309 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:36.589404 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af5b8f-3f8d-473f-943d-28bf993ff994 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:36.625786 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc98844-51fc-4b59-aaab-dd4bfc6f84b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:36.626451 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc98844-51fc-4b59-aaab-dd4bfc6f84b8 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:36.626860 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc98844-51fc-4b59-aaab-dd4bfc6f84b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:36.627138 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc98844-51fc-4b59-aaab-dd4bfc6f84b8 None admin] PATH_INFO: `/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:36.628347 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc98844-51fc-4b59-aaab-dd4bfc6f84b8 None admin] RBAC: Authorizing `identity:get_project(project_id=759a514c6fa54b4b8fa90f2d4f181f99)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:36.630424 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc98844-51fc-4b59-aaab-dd4bfc6f84b8 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:36.632509 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 207/407] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:12:36 2022] GET /identity/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:36.633272 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc98844-51fc-4b59-aaab-dd4bfc6f84b8 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:36.650952 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0098e0-4f0b-41da-9054-cf6713a5e3e4 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:36.688723 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a32607-8adb-4e12-8607-0b0e6a364fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:36.689614 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a32607-8adb-4e12-8607-0b0e6a364fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:36.689998 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a32607-8adb-4e12-8607-0b0e6a364fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:36.690381 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a32607-8adb-4e12-8607-0b0e6a364fa3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:36.702221 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a32607-8adb-4e12-8607-0b0e6a364fa3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:36.708262 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-72a32607-8adb-4e12-8607-0b0e6a364fa3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:12:36.708262 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:36.708262 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:36.708262 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:36.708262 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:36.708262 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:36.708262 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:36.708262 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:36.708262 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:36.708262 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:36.708262 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:36.708262 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:36.708262 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:36.708262 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:36.708262 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:36.708262 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:36.710036 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:36.710036 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:36.710036 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:36.710036 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:36.710036 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:36.710036 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:36.710036 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:36.710036 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:36.710036 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:36.710036 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:36.710036 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:36.710036 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:36.710036 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:36.710036 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:36.710036 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:36.710036 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:36.711945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:36.711945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:36.711945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:36.711945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:36.711945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:36.711945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:36.711945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:36.711945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:36.711945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:36.711945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:36.711945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:12:36.711945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:36.711945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 201/408] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 15:12:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:36.711945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a32607-8adb-4e12-8607-0b0e6a364fa3 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:36.745297 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc98844-51fc-4b59-aaab-dd4bfc6f84b8 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:36.763606 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:36.782166 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2130600a-c9b8-49c4-8012-143059d53102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:36.782862 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130600a-c9b8-49c4-8012-143059d53102 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:36.783141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130600a-c9b8-49c4-8012-143059d53102 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:36.783447 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130600a-c9b8-49c4-8012-143059d53102 None admin] PATH_INFO: `/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:36.784812 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2130600a-c9b8-49c4-8012-143059d53102 None admin] RBAC: Authorizing `identity:get_project(project_id=759a514c6fa54b4b8fa90f2d4f181f99)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:36.786848 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2130600a-c9b8-49c4-8012-143059d53102 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:36.788718 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 208/409] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:12:36 2022] GET /identity/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:36.789823 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2130600a-c9b8-49c4-8012-143059d53102 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:36.814946 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:36.830406 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a32607-8adb-4e12-8607-0b0e6a364fa3 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:36.875723 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41cb42e-c4ff-47b7-aaeb-d643847fb52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:36.876712 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41cb42e-c4ff-47b7-aaeb-d643847fb52a None admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:36.877162 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41cb42e-c4ff-47b7-aaeb-d643847fb52a None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:36.877597 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41cb42e-c4ff-47b7-aaeb-d643847fb52a None admin] PATH_INFO: `/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99/groups/02515ae8bab14d9386d9b69742479fd2/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:36.879830 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41cb42e-c4ff-47b7-aaeb-d643847fb52a None admin] RBAC: Authorizing `identity:create_grant(project_id=759a514c6fa54b4b8fa90f2d4f181f99, group_id=02515ae8bab14d9386d9b69742479fd2, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:36.881528 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41cb42e-c4ff-47b7-aaeb-d643847fb52a None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:36.905721 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2130600a-c9b8-49c4-8012-143059d53102 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:36.942507 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a001daf4-ad31-412b-bd2d-63bda239e70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:36.943122 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a001daf4-ad31-412b-bd2d-63bda239e70e None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:36.943455 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a001daf4-ad31-412b-bd2d-63bda239e70e None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:36.943739 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a001daf4-ad31-412b-bd2d-63bda239e70e None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:36.944480 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a001daf4-ad31-412b-bd2d-63bda239e70e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:36.944893 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a001daf4-ad31-412b-bd2d-63bda239e70e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:36.946152 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a001daf4-ad31-412b-bd2d-63bda239e70e None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:36.951561 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 202/410] 10.222.0.40 () {58 vars in 1442 bytes} [Fri Aug 5 15:12:36 2022] PUT /identity/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99/groups/02515ae8bab14d9386d9b69742479fd2/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:12:36.952847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41cb42e-c4ff-47b7-aaeb-d643847fb52a None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:36.964982 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 209/411] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:12:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:36.970346 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a001daf4-ad31-412b-bd2d-63bda239e70e None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:37.070841 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:37.087369 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41cb42e-c4ff-47b7-aaeb-d643847fb52a None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:37.132702 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccac205-3b2f-48ca-96af-9a5324ba8c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:37.134965 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccac205-3b2f-48ca-96af-9a5324ba8c66 None admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:37.135482 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccac205-3b2f-48ca-96af-9a5324ba8c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:37.135958 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccac205-3b2f-48ca-96af-9a5324ba8c66 None admin] PATH_INFO: `/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99/groups/02515ae8bab14d9386d9b69742479fd2/roles/944427ca14ee4c81a05ec67e54f50e7f` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:37.138622 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccac205-3b2f-48ca-96af-9a5324ba8c66 None admin] RBAC: Authorizing `identity:create_grant(project_id=759a514c6fa54b4b8fa90f2d4f181f99, group_id=02515ae8bab14d9386d9b69742479fd2, role_id=944427ca14ee4c81a05ec67e54f50e7f)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:37.140430 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccac205-3b2f-48ca-96af-9a5324ba8c66 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:37.189254 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 203/412] 10.222.0.40 () {58 vars in 1442 bytes} [Fri Aug 5 15:12:36 2022] PUT /identity/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99/groups/02515ae8bab14d9386d9b69742479fd2/roles/944427ca14ee4c81a05ec67e54f50e7f => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:12:37.198040 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a001daf4-ad31-412b-bd2d-63bda239e70e None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:37.221753 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc749aa-4329-4012-9894-08a5c7cb234d None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:37.222056 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc749aa-4329-4012-9894-08a5c7cb234d None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:37.222332 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc749aa-4329-4012-9894-08a5c7cb234d None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:37.224719 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 204/413] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:37.229920 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee3e4a-40ed-4350-81bb-fb368d1c92f4 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:37.230216 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee3e4a-40ed-4350-81bb-fb368d1c92f4 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:37.230626 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee3e4a-40ed-4350-81bb-fb368d1c92f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:37.243575 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d354749-9dde-45c0-8004-d8beb534e646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:37.244595 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d354749-9dde-45c0-8004-d8beb534e646 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:37.245025 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d354749-9dde-45c0-8004-d8beb534e646 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:37.245448 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d354749-9dde-45c0-8004-d8beb534e646 None admin] PATH_INFO: `/v3/groups/d178fe555c6d4070adcb0068db625fa9` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:37.246786 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d354749-9dde-45c0-8004-d8beb534e646 None admin] RBAC: Authorizing `identity:get_group(group_id=d178fe555c6d4070adcb0068db625fa9)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:37.248452 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d354749-9dde-45c0-8004-d8beb534e646 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:37.250822 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 210/414] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 5 15:12:36 2022] GET /identity/v3/groups/d178fe555c6d4070adcb0068db625fa9 => generated 229 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:37.257877 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d354749-9dde-45c0-8004-d8beb534e646 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:37.285304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-f0ee3e4a-40ed-4350-81bb-fb368d1c92f4 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:37.287158 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-f0ee3e4a-40ed-4350-81bb-fb368d1c92f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:37.289507 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:37.374999 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d354749-9dde-45c0-8004-d8beb534e646 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:37.394585 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:37.417259 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 205/415] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 15:12:37.421185 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15555c81-56d3-4cb4-b8c3-7c4483e66167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:37.422140 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15555c81-56d3-4cb4-b8c3-7c4483e66167 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:37.422664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15555c81-56d3-4cb4-b8c3-7c4483e66167 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:37.423144 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15555c81-56d3-4cb4-b8c3-7c4483e66167 None admin] PATH_INFO: `/v3/projects/0744d3ce240a4f198b94f2d1e6d62c8e` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:37.424558 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15555c81-56d3-4cb4-b8c3-7c4483e66167 None admin] RBAC: Authorizing `identity:get_project(project_id=0744d3ce240a4f198b94f2d1e6d62c8e)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:37.426314 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15555c81-56d3-4cb4-b8c3-7c4483e66167 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:37.428599 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 211/416] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:12:37 2022] GET /identity/v3/projects/0744d3ce240a4f198b94f2d1e6d62c8e => generated 344 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:37.434219 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee3e4a-40ed-4350-81bb-fb368d1c92f4 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:37.447597 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15555c81-56d3-4cb4-b8c3-7c4483e66167 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:37.580842 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee3e4a-40ed-4350-81bb-fb368d1c92f4 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:37.619045 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50794460-5d5e-4c95-bd46-dd4b6d738965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:37.620159 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50794460-5d5e-4c95-bd46-dd4b6d738965 None admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:37.620651 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50794460-5d5e-4c95-bd46-dd4b6d738965 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:37.621199 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50794460-5d5e-4c95-bd46-dd4b6d738965 None admin] PATH_INFO: `/v3/projects/0744d3ce240a4f198b94f2d1e6d62c8e/groups/d178fe555c6d4070adcb0068db625fa9/roles/7c88c815db854954b516ab97cd471647` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:37.623510 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50794460-5d5e-4c95-bd46-dd4b6d738965 None admin] RBAC: Authorizing `identity:create_grant(project_id=0744d3ce240a4f198b94f2d1e6d62c8e, group_id=d178fe555c6d4070adcb0068db625fa9, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:37.625516 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50794460-5d5e-4c95-bd46-dd4b6d738965 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:37.638267 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15555c81-56d3-4cb4-b8c3-7c4483e66167 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:37.679096 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8460ef3b-2433-469b-9f6c-080fea166bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:37.680139 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460ef3b-2433-469b-9f6c-080fea166bdd None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:37.680654 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460ef3b-2433-469b-9f6c-080fea166bdd None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:37.681134 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460ef3b-2433-469b-9f6c-080fea166bdd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:37.691722 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:37.698630 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 206/417] 10.222.0.40 () {58 vars in 1442 bytes} [Fri Aug 5 15:12:37 2022] PUT /identity/v3/projects/0744d3ce240a4f198b94f2d1e6d62c8e/groups/d178fe555c6d4070adcb0068db625fa9/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:12:37.703419 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8460ef3b-2433-469b-9f6c-080fea166bdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:37.710783 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-8460ef3b-2433-469b-9f6c-080fea166bdd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 15:12:37.710783 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:37.710783 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:37.710783 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:37.710783 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:37.710783 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:37.710783 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:37.710783 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:37.710783 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:37.710783 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:37.710783 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:37.710783 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:37.710783 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:37.710783 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:37.710783 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:37.710783 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:37.712012 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:37.712012 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:37.712012 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:37.712012 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:37.712012 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:37.712012 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:37.712012 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:37.712012 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:37.712012 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:37.712012 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:37.712012 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:37.712012 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:37.712012 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:37.712012 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:37.712012 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:37.712012 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:37.712884 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:37.712884 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:37.712884 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:37.712884 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:37.712884 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:37.712884 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:37.712884 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:37.712884 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:37.712884 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:37.712884 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:37.712884 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 15:12:37.712884 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:37.715621 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 212/418] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:12:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:12:37.721552 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50794460-5d5e-4c95-bd46-dd4b6d738965 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:37.880657 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e727a-4c65-4e72-aa04-74daf1d5528f None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:37.880990 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e727a-4c65-4e72-aa04-74daf1d5528f None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:37.881205 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e727a-4c65-4e72-aa04-74daf1d5528f None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:37.883930 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 213/419] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 15:12:37.885476 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50794460-5d5e-4c95-bd46-dd4b6d738965 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:37.888456 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549de97-8e68-4b12-ad07-66ed61ab9410 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:37.888743 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549de97-8e68-4b12-ad07-66ed61ab9410 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:37.889021 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549de97-8e68-4b12-ad07-66ed61ab9410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:37.908412 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-6549de97-8e68-4b12-ad07-66ed61ab9410 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:37.910079 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-6549de97-8e68-4b12-ad07-66ed61ab9410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:37.938561 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeaa822-fad2-4e09-b6a4-714e5216be7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:37.939228 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeaa822-fad2-4e09-b6a4-714e5216be7a None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:37.939589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeaa822-fad2-4e09-b6a4-714e5216be7a None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:37.940146 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeaa822-fad2-4e09-b6a4-714e5216be7a None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:37.940931 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeaa822-fad2-4e09-b6a4-714e5216be7a None admin] RBAC: Adding query filter params (name=member) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:37.941312 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeaa822-fad2-4e09-b6a4-714e5216be7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:37.943351 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeaa822-fad2-4e09-b6a4-714e5216be7a None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:37.953278 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 207/420] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 15:12:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:37.958426 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeaa822-fad2-4e09-b6a4-714e5216be7a None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:37.963249 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:37.999608 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 214/421] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 15:12:38.031868 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:38.042179 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6549de97-8e68-4b12-ad07-66ed61ab9410 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:38.132725 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeaa822-fad2-4e09-b6a4-714e5216be7a None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:38.197798 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beaa920-562c-443b-ae3f-c3e6079455bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:38.198345 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beaa920-562c-443b-ae3f-c3e6079455bc None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:38.198630 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beaa920-562c-443b-ae3f-c3e6079455bc None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:38.198906 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beaa920-562c-443b-ae3f-c3e6079455bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:38.215729 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6549de97-8e68-4b12-ad07-66ed61ab9410 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:38.278826 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab04a3-2c7e-4521-92a4-d8659bc41aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:38.279475 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab04a3-2c7e-4521-92a4-d8659bc41aea None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:38.279762 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab04a3-2c7e-4521-92a4-d8659bc41aea None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:38.280039 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab04a3-2c7e-4521-92a4-d8659bc41aea None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:38.300182 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab04a3-2c7e-4521-92a4-d8659bc41aea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:38.305860 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-c9ab04a3-2c7e-4521-92a4-d8659bc41aea None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 15:12:38.305860 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:38.305860 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:38.305860 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:38.305860 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:38.305860 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:38.305860 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:38.305860 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:38.305860 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:38.305860 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:38.305860 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:38.305860 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:38.305860 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:38.305860 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:38.305860 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:38.305860 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:38.306761 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:38.306761 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:38.306761 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:38.306761 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:38.306761 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:38.306761 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:38.306761 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:38.306761 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:38.306761 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:38.306761 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:38.306761 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:38.306761 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:38.306761 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:38.306761 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:38.306761 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:38.306761 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:38.307653 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:38.307653 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:38.307653 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:38.307653 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:38.307653 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:38.307653 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:38.307653 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:38.307653 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:38.307653 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:38.307653 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:38.307653 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 15:12:38.307653 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:38.307653 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 215/422] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 15:12:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:12:38.314760 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab04a3-2c7e-4521-92a4-d8659bc41aea None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:38.324415 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beaa920-562c-443b-ae3f-c3e6079455bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:38.325912 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beaa920-562c-443b-ae3f-c3e6079455bc None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:38.372450 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:38.446644 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:38.514725 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab04a3-2c7e-4521-92a4-d8659bc41aea None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:38.529730 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 208/423] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:12:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 15:12:38.538888 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beaa920-562c-443b-ae3f-c3e6079455bc None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:38.584204 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c4a276-ceb7-493a-97f5-008fb2cad43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:38.586546 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4a276-ceb7-493a-97f5-008fb2cad43b None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:38.587252 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4a276-ceb7-493a-97f5-008fb2cad43b None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:38.588842 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4a276-ceb7-493a-97f5-008fb2cad43b None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:38.589821 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c4a276-ceb7-493a-97f5-008fb2cad43b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:38.591237 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c4a276-ceb7-493a-97f5-008fb2cad43b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:38.593417 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c4a276-ceb7-493a-97f5-008fb2cad43b None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:38.613625 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 216/424] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:12:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:38.619337 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c4a276-ceb7-493a-97f5-008fb2cad43b None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:38.671351 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beaa920-562c-443b-ae3f-c3e6079455bc None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:38.708569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657090b-3f67-4f16-a547-c2348b1df425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:38.709556 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657090b-3f67-4f16-a547-c2348b1df425 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:38.710040 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657090b-3f67-4f16-a547-c2348b1df425 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:38.710536 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657090b-3f67-4f16-a547-c2348b1df425 None admin] PATH_INFO: `/v3/projects/b972ca1ff8ee4c558eeb45ec24303061` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:38.711916 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2657090b-3f67-4f16-a547-c2348b1df425 None admin] RBAC: Authorizing `identity:get_project(project_id=b972ca1ff8ee4c558eeb45ec24303061)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:38.713618 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2657090b-3f67-4f16-a547-c2348b1df425 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:38.716008 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 209/425] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:12:38 2022] GET /identity/v3/projects/b972ca1ff8ee4c558eeb45ec24303061 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:38.721000 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657090b-3f67-4f16-a547-c2348b1df425 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:38.752305 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:38.797040 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:38.804223 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c4a276-ceb7-493a-97f5-008fb2cad43b None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:38.839433 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8787c5bf-7971-4c94-b561-bb6f5ee29aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:38.840053 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8787c5bf-7971-4c94-b561-bb6f5ee29aec None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:38.840362 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8787c5bf-7971-4c94-b561-bb6f5ee29aec None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:38.840649 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8787c5bf-7971-4c94-b561-bb6f5ee29aec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:38.845421 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657090b-3f67-4f16-a547-c2348b1df425 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:38.881730 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81237af-0cab-4967-a15c-bd1eb4db0a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:38.882794 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81237af-0cab-4967-a15c-bd1eb4db0a66 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:38.883291 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81237af-0cab-4967-a15c-bd1eb4db0a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:38.883808 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81237af-0cab-4967-a15c-bd1eb4db0a66 None admin] PATH_INFO: `/v3/groups/02515ae8bab14d9386d9b69742479fd2` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:38.885247 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81237af-0cab-4967-a15c-bd1eb4db0a66 None admin] RBAC: Authorizing `identity:get_group(group_id=02515ae8bab14d9386d9b69742479fd2)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:38.886958 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81237af-0cab-4967-a15c-bd1eb4db0a66 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:38.889232 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 210/426] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 5 15:12:38 2022] GET /identity/v3/groups/02515ae8bab14d9386d9b69742479fd2 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:38.896661 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81237af-0cab-4967-a15c-bd1eb4db0a66 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:38.970421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8787c5bf-7971-4c94-b561-bb6f5ee29aec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:38.971917 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8787c5bf-7971-4c94-b561-bb6f5ee29aec None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:39.059222 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:39.067884 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81237af-0cab-4967-a15c-bd1eb4db0a66 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:39.107211 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac283ad-726b-42dc-a3a2-450b6fa95558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:39.107846 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac283ad-726b-42dc-a3a2-450b6fa95558 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:39.108227 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac283ad-726b-42dc-a3a2-450b6fa95558 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:39.108514 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac283ad-726b-42dc-a3a2-450b6fa95558 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:39.109438 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac283ad-726b-42dc-a3a2-450b6fa95558 None admin] RBAC: Adding query filter params (group.id=02515ae8bab14d9386d9b69742479fd2, role.id=d6c91838e4944cfbb23e1b6ecb009d26, scope.project.id=b972ca1ff8ee4c558eeb45ec24303061) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:39.109689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac283ad-726b-42dc-a3a2-450b6fa95558 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:39.111193 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac283ad-726b-42dc-a3a2-450b6fa95558 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:39.118959 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 211/427] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Aug 5 15:12:38 2022] GET /identity/v3/role_assignments?group.id=02515ae8bab14d9386d9b69742479fd2&scope.project.id=b972ca1ff8ee4c558eeb45ec24303061&role.id=d6c91838e4944cfbb23e1b6ecb009d26 => generated 611 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:39.163063 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:39.178441 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 217/428] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:12:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 15:12:39.184401 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac283ad-726b-42dc-a3a2-450b6fa95558 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:39.333764 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac283ad-726b-42dc-a3a2-450b6fa95558 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:39.379243 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d8cc71-ce3f-48ea-8fb3-7e429355a91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:39.380511 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d8cc71-ce3f-48ea-8fb3-7e429355a91b None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:39.380798 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d8cc71-ce3f-48ea-8fb3-7e429355a91b None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:39.381134 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d8cc71-ce3f-48ea-8fb3-7e429355a91b None admin] PATH_INFO: `/v3/projects/b972ca1ff8ee4c558eeb45ec24303061` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:39.382349 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d8cc71-ce3f-48ea-8fb3-7e429355a91b None admin] RBAC: Authorizing `identity:get_project(project_id=b972ca1ff8ee4c558eeb45ec24303061)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:39.383801 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d8cc71-ce3f-48ea-8fb3-7e429355a91b None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:39.385832 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 212/429] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:12:39 2022] GET /identity/v3/projects/b972ca1ff8ee4c558eeb45ec24303061 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:39.391134 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8787c5bf-7971-4c94-b561-bb6f5ee29aec None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:39.487582 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2729a64-a45c-4aba-bc01-b89f17f9a208 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:39.487914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2729a64-a45c-4aba-bc01-b89f17f9a208 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:39.488190 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2729a64-a45c-4aba-bc01-b89f17f9a208 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:39.490348 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 213/430] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:39.496850 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d04f2-1a5e-49cc-9423-cd0e0613bfd6 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:39.497143 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d04f2-1a5e-49cc-9423-cd0e0613bfd6 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:39.497430 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d04f2-1a5e-49cc-9423-cd0e0613bfd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:39.500028 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8787c5bf-7971-4c94-b561-bb6f5ee29aec None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:39.516946 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-a63d04f2-1a5e-49cc-9423-cd0e0613bfd6 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:39.521225 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-a63d04f2-1a5e-49cc-9423-cd0e0613bfd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:39.538032 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24649abc-f4c4-4557-a4f4-55b2da79c3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:39.538723 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24649abc-f4c4-4557-a4f4-55b2da79c3b2 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:39.539078 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24649abc-f4c4-4557-a4f4-55b2da79c3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:39.539362 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24649abc-f4c4-4557-a4f4-55b2da79c3b2 None admin] PATH_INFO: `/v3/groups/02515ae8bab14d9386d9b69742479fd2` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:39.540582 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24649abc-f4c4-4557-a4f4-55b2da79c3b2 None admin] RBAC: Authorizing `identity:get_group(group_id=02515ae8bab14d9386d9b69742479fd2)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:39.542136 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24649abc-f4c4-4557-a4f4-55b2da79c3b2 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:39.544088 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 218/431] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 5 15:12:39 2022] GET /identity/v3/groups/02515ae8bab14d9386d9b69742479fd2 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:12:39.546104 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:39.549571 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24649abc-f4c4-4557-a4f4-55b2da79c3b2 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:39.598477 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 214/432] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:39.624964 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d04f2-1a5e-49cc-9423-cd0e0613bfd6 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:39.625173 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:39.684595 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24649abc-f4c4-4557-a4f4-55b2da79c3b2 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:39.730924 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72723a9f-e4de-4f56-824d-464c4114f862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:39.732118 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72723a9f-e4de-4f56-824d-464c4114f862 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:39.732408 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72723a9f-e4de-4f56-824d-464c4114f862 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:39.732688 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72723a9f-e4de-4f56-824d-464c4114f862 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:39.733977 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72723a9f-e4de-4f56-824d-464c4114f862 None admin] RBAC: Adding query filter params (group.id=02515ae8bab14d9386d9b69742479fd2, role.id=944427ca14ee4c81a05ec67e54f50e7f, scope.project.id=b972ca1ff8ee4c558eeb45ec24303061) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:39.734279 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72723a9f-e4de-4f56-824d-464c4114f862 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:39.736175 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72723a9f-e4de-4f56-824d-464c4114f862 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:39.743709 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 219/433] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Aug 5 15:12:39 2022] GET /identity/v3/role_assignments?group.id=02515ae8bab14d9386d9b69742479fd2&scope.project.id=b972ca1ff8ee4c558eeb45ec24303061&role.id=944427ca14ee4c81a05ec67e54f50e7f => generated 611 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:12:39.744960 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de5179-e760-4a14-b817-226589b97899 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:39.745252 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de5179-e760-4a14-b817-226589b97899 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:39.745533 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de5179-e760-4a14-b817-226589b97899 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:39.746938 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 220/434] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:39.752949 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a983e-a738-4978-a8bf-3f949268fcc0 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:39.753242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a983e-a738-4978-a8bf-3f949268fcc0 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:39.753526 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a983e-a738-4978-a8bf-3f949268fcc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:39.773696 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-513a983e-a738-4978-a8bf-3f949268fcc0 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:39.774096 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d04f2-1a5e-49cc-9423-cd0e0613bfd6 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:39.775449 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-513a983e-a738-4978-a8bf-3f949268fcc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:39.823753 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a443fe6-b965-4e7c-8071-ecd8d9a45da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:39.824834 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a443fe6-b965-4e7c-8071-ecd8d9a45da4 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:39.825346 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a443fe6-b965-4e7c-8071-ecd8d9a45da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:39.825825 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a443fe6-b965-4e7c-8071-ecd8d9a45da4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:39.844315 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a443fe6-b965-4e7c-8071-ecd8d9a45da4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:39.850154 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-1a443fe6-b965-4e7c-8071-ecd8d9a45da4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 15:12:39.850154 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:39.850154 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:39.850154 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:39.850154 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:39.850154 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:39.850154 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:39.850154 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:39.850154 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:39.850154 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:39.850154 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:39.850154 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:39.850154 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:39.850154 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:39.850154 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:39.850154 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:39.851441 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:39.851441 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:39.851441 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:39.851441 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:39.851441 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:39.851441 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:39.851441 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:39.851441 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:39.851441 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:39.851441 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:39.851441 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:39.851441 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:39.851441 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:39.851441 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:39.851441 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:39.851441 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:39.860892 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:39.860892 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:39.860892 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:39.860892 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:39.860892 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:39.860892 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:39.860892 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:39.860892 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:39.860892 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:39.860892 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:39.860892 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 15:12:39.860892 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:39.860892 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 215/435] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:12:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:39.861955 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a443fe6-b965-4e7c-8071-ecd8d9a45da4 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:39.899117 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:39.901220 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 221/436] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 15:12:39.923946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a983e-a738-4978-a8bf-3f949268fcc0 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:39.993377 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a443fe6-b965-4e7c-8071-ecd8d9a45da4 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:40.015851 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:40.050538 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149c2ec-c13b-43ff-b136-0c65266c6bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:40.052104 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149c2ec-c13b-43ff-b136-0c65266c6bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:40.052613 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149c2ec-c13b-43ff-b136-0c65266c6bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:40.053087 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149c2ec-c13b-43ff-b136-0c65266c6bc3 None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:40.054482 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a149c2ec-c13b-43ff-b136-0c65266c6bc3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:40.055227 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a149c2ec-c13b-43ff-b136-0c65266c6bc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:40.058358 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a149c2ec-c13b-43ff-b136-0c65266c6bc3 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:40.069481 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a983e-a738-4978-a8bf-3f949268fcc0 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:40.069974 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 216/437] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 15:12:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:40.075371 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149c2ec-c13b-43ff-b136-0c65266c6bc3 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:40.114159 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7dd8a0-8135-4b94-8c08-a8ff24240bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:40.114746 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7dd8a0-8135-4b94-8c08-a8ff24240bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:40.115032 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7dd8a0-8135-4b94-8c08-a8ff24240bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:40.115304 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7dd8a0-8135-4b94-8c08-a8ff24240bc3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:40.133980 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7dd8a0-8135-4b94-8c08-a8ff24240bc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:40.138965 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-7d7dd8a0-8135-4b94-8c08-a8ff24240bc3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 15:12:40.138965 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:40.138965 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:40.138965 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:40.138965 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:40.138965 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:40.138965 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:40.138965 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:40.138965 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:40.138965 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:40.138965 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:40.138965 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:40.138965 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:40.138965 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:40.138965 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:40.138965 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:40.139871 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:40.139871 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:40.139871 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:40.139871 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:40.139871 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:40.139871 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:40.139871 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:40.139871 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:40.139871 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:40.139871 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:40.139871 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:40.139871 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:40.139871 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:40.139871 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:40.139871 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:40.139871 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:40.140737 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:40.140737 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:40.140737 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:40.140737 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:40.140737 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:40.140737 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:40.140737 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:40.140737 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:40.140737 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:40.140737 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:40.140737 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 15:12:40.140737 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:40.141374 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 222/438] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 15:12:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:40.146120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7dd8a0-8135-4b94-8c08-a8ff24240bc3 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:40.191558 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:40.215682 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149c2ec-c13b-43ff-b136-0c65266c6bc3 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:40.254099 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ba055b-0979-4b02-b8e5-7f3cfea9eebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:40.255901 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba055b-0979-4b02-b8e5-7f3cfea9eebb None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:40.255901 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba055b-0979-4b02-b8e5-7f3cfea9eebb None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:40.255901 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba055b-0979-4b02-b8e5-7f3cfea9eebb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:40.276125 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7dd8a0-8135-4b94-8c08-a8ff24240bc3 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:40.292267 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:40.310472 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c7a7d9-73df-454c-8deb-9edd70ffd096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:40.311087 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c7a7d9-73df-454c-8deb-9edd70ffd096 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:40.311374 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c7a7d9-73df-454c-8deb-9edd70ffd096 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:40.311657 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c7a7d9-73df-454c-8deb-9edd70ffd096 None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:40.312499 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c7a7d9-73df-454c-8deb-9edd70ffd096 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:40.312809 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c7a7d9-73df-454c-8deb-9edd70ffd096 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:40.314190 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c7a7d9-73df-454c-8deb-9edd70ffd096 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:40.323104 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 223/439] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:12:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:12:40.324364 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3f1b0-738c-4a1e-a185-bbec61e30a62 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:40.324658 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3f1b0-738c-4a1e-a185-bbec61e30a62 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:40.324935 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3f1b0-738c-4a1e-a185-bbec61e30a62 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:40.325710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ba055b-0979-4b02-b8e5-7f3cfea9eebb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:40.326150 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 224/440] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:40.327570 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ba055b-0979-4b02-b8e5-7f3cfea9eebb None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:40.328926 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a3f1b0-738c-4a1e-a185-bbec61e30a62 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:40.409138 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:40.429007 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 217/441] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:12:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 15:12:40.429212 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a3f1b0-738c-4a1e-a185-bbec61e30a62 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:40.430216 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c25e9f-e061-4f59-adda-677cbee933d9 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:40.430507 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c25e9f-e061-4f59-adda-677cbee933d9 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:40.430777 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c25e9f-e061-4f59-adda-677cbee933d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:40.448224 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-49c25e9f-e061-4f59-adda-677cbee933d9 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:40.449424 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-49c25e9f-e061-4f59-adda-677cbee933d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:40.467032 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ebc601-6083-4344-bae6-5db6e344795b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:40.467650 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ebc601-6083-4344-bae6-5db6e344795b None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:40.467930 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ebc601-6083-4344-bae6-5db6e344795b None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:40.468245 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ebc601-6083-4344-bae6-5db6e344795b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:40.514635 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:40.517442 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 218/442] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:40.518055 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c25e9f-e061-4f59-adda-677cbee933d9 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:40.553513 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ebc601-6083-4344-bae6-5db6e344795b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:40.554979 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ebc601-6083-4344-bae6-5db6e344795b None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:40.615398 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c25e9f-e061-4f59-adda-677cbee933d9 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:40.626286 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:40.649346 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b1fe07-b15c-4cdd-934a-9e5c3723206e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:40.649989 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b1fe07-b15c-4cdd-934a-9e5c3723206e None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:40.650271 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b1fe07-b15c-4cdd-934a-9e5c3723206e None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:40.650553 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b1fe07-b15c-4cdd-934a-9e5c3723206e None admin] PATH_INFO: `/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:40.651713 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b1fe07-b15c-4cdd-934a-9e5c3723206e None admin] RBAC: Authorizing `identity:get_project(project_id=759a514c6fa54b4b8fa90f2d4f181f99)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:40.653171 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b1fe07-b15c-4cdd-934a-9e5c3723206e None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:40.654374 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 225/443] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:12:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 15:12:40.656113 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 219/444] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:12:40 2022] GET /identity/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99 => generated 302 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:40.656247 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ebc601-6083-4344-bae6-5db6e344795b None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:40.660214 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b1fe07-b15c-4cdd-934a-9e5c3723206e None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:40.730945 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:40.752094 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ebc601-6083-4344-bae6-5db6e344795b None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:40.758194 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b1fe07-b15c-4cdd-934a-9e5c3723206e None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:40.784928 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041ab1d-997f-426d-b54b-6aa356f8e1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:40.786085 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041ab1d-997f-426d-b54b-6aa356f8e1e0 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:40.786366 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041ab1d-997f-426d-b54b-6aa356f8e1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:40.786632 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041ab1d-997f-426d-b54b-6aa356f8e1e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:40.791664 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65538b04-e5c9-41ee-9fed-1e04ef3f02be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:40.792346 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65538b04-e5c9-41ee-9fed-1e04ef3f02be None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:40.792636 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65538b04-e5c9-41ee-9fed-1e04ef3f02be None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:40.792919 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65538b04-e5c9-41ee-9fed-1e04ef3f02be None admin] PATH_INFO: `/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:40.794091 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65538b04-e5c9-41ee-9fed-1e04ef3f02be None admin] RBAC: Authorizing `identity:get_project(project_id=759a514c6fa54b4b8fa90f2d4f181f99)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:40.795538 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65538b04-e5c9-41ee-9fed-1e04ef3f02be None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:40.796574 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8041ab1d-997f-426d-b54b-6aa356f8e1e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:40.797248 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 220/445] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:12:40 2022] GET /identity/v3/projects/759a514c6fa54b4b8fa90f2d4f181f99 => generated 302 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:12:40.798015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65538b04-e5c9-41ee-9fed-1e04ef3f02be None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:40.802586 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-8041ab1d-997f-426d-b54b-6aa356f8e1e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:12:40.802586 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:40.802586 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:40.802586 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:40.802586 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:40.802586 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:40.802586 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:40.802586 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:40.802586 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:40.802586 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:40.802586 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:40.802586 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:40.802586 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:40.802586 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:40.802586 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:40.802586 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:40.803713 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:40.803713 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:40.803713 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:40.803713 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:40.803713 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:40.803713 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:40.803713 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:40.803713 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:40.803713 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:40.803713 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:40.803713 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:40.803713 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:40.803713 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:40.803713 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:40.803713 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:40.803713 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:40.804559 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:40.804559 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:40.804559 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:40.804559 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:40.804559 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:40.804559 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:40.804559 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:40.804559 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:40.804559 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:40.804559 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:40.804559 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:12:40.804559 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:40.804559 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 226/446] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 15:12:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:40.804559 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041ab1d-997f-426d-b54b-6aa356f8e1e0 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:40.854788 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:40.904516 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041ab1d-997f-426d-b54b-6aa356f8e1e0 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:40.907437 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65538b04-e5c9-41ee-9fed-1e04ef3f02be None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:40.938070 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5470a8-2880-4470-a3d4-25e98d03a3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:40.938759 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5470a8-2880-4470-a3d4-25e98d03a3e5 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:40.939129 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5470a8-2880-4470-a3d4-25e98d03a3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:40.939411 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5470a8-2880-4470-a3d4-25e98d03a3e5 None admin] PATH_INFO: `/v3/groups/02515ae8bab14d9386d9b69742479fd2` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:40.940621 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5470a8-2880-4470-a3d4-25e98d03a3e5 None admin] RBAC: Authorizing `identity:get_group(group_id=02515ae8bab14d9386d9b69742479fd2)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:40.941498 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65242a-62a2-4a69-bba4-48d91cb6623c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:40.942133 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65242a-62a2-4a69-bba4-48d91cb6623c None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:40.942353 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5470a8-2880-4470-a3d4-25e98d03a3e5 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:40.942446 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65242a-62a2-4a69-bba4-48d91cb6623c None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:40.942697 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65242a-62a2-4a69-bba4-48d91cb6623c None admin] PATH_INFO: `/v3/groups/02515ae8bab14d9386d9b69742479fd2` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:40.943857 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb65242a-62a2-4a69-bba4-48d91cb6623c None admin] RBAC: Authorizing `identity:get_group(group_id=02515ae8bab14d9386d9b69742479fd2)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:40.943990 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 227/447] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 5 15:12:40 2022] GET /identity/v3/groups/02515ae8bab14d9386d9b69742479fd2 => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:40.944770 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5470a8-2880-4470-a3d4-25e98d03a3e5 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:40.945343 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb65242a-62a2-4a69-bba4-48d91cb6623c None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:40.951512 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 221/448] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 5 15:12:40 2022] GET /identity/v3/groups/02515ae8bab14d9386d9b69742479fd2 => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:40.951512 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65242a-62a2-4a69-bba4-48d91cb6623c None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:40.953876 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:41.043454 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5470a8-2880-4470-a3d4-25e98d03a3e5 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:41.049682 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65242a-62a2-4a69-bba4-48d91cb6623c None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:41.064351 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:41.075336 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf5dfe8-ae75-4d20-b253-4c39827d5a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:41.080772 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5dfe8-ae75-4d20-b253-4c39827d5a51 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:41.080772 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5dfe8-ae75-4d20-b253-4c39827d5a51 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:41.080772 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5dfe8-ae75-4d20-b253-4c39827d5a51 None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:41.080772 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf5dfe8-ae75-4d20-b253-4c39827d5a51 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:41.080772 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf5dfe8-ae75-4d20-b253-4c39827d5a51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:41.080772 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf5dfe8-ae75-4d20-b253-4c39827d5a51 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:41.084429 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca2636-f85f-47c1-bced-62757b24ce8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:41.085039 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca2636-f85f-47c1-bced-62757b24ce8e None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:41.085328 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca2636-f85f-47c1-bced-62757b24ce8e None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:41.085609 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca2636-f85f-47c1-bced-62757b24ce8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:41.086403 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca2636-f85f-47c1-bced-62757b24ce8e None admin] RBAC: Adding query filter params (group.id=02515ae8bab14d9386d9b69742479fd2, role.id=944427ca14ee4c81a05ec67e54f50e7f, scope.project.id=759a514c6fa54b4b8fa90f2d4f181f99) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:41.086723 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca2636-f85f-47c1-bced-62757b24ce8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:41.087325 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 228/449] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:12:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:41.088120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf5dfe8-ae75-4d20-b253-4c39827d5a51 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:41.088289 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca2636-f85f-47c1-bced-62757b24ce8e None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:41.103614 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 222/450] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Aug 5 15:12:40 2022] GET /identity/v3/role_assignments?group.id=02515ae8bab14d9386d9b69742479fd2&scope.project.id=759a514c6fa54b4b8fa90f2d4f181f99&role.id=944427ca14ee4c81a05ec67e54f50e7f => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:12:41.103614 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca2636-f85f-47c1-bced-62757b24ce8e None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:41.175681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:41.214672 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf5dfe8-ae75-4d20-b253-4c39827d5a51 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:41.229647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca2636-f85f-47c1-bced-62757b24ce8e None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:41.256913 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afabf99-c235-4bbc-ab12-a5914818ae17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:41.257440 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afabf99-c235-4bbc-ab12-a5914818ae17 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:41.257738 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afabf99-c235-4bbc-ab12-a5914818ae17 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:41.258020 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afabf99-c235-4bbc-ab12-a5914818ae17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:41.259357 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afabf99-c235-4bbc-ab12-a5914818ae17 None admin] RBAC: Adding query filter params (group.id=02515ae8bab14d9386d9b69742479fd2, role.id=d6c91838e4944cfbb23e1b6ecb009d26, scope.project.id=759a514c6fa54b4b8fa90f2d4f181f99) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:41.259357 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afabf99-c235-4bbc-ab12-a5914818ae17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:41.260642 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afabf99-c235-4bbc-ab12-a5914818ae17 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:41.269267 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 229/451] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Aug 5 15:12:41 2022] GET /identity/v3/role_assignments?group.id=02515ae8bab14d9386d9b69742479fd2&scope.project.id=759a514c6fa54b4b8fa90f2d4f181f99&role.id=d6c91838e4944cfbb23e1b6ecb009d26 => generated 611 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:41.272208 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3720aba-c7c7-49e0-8119-375a6a086f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:41.272854 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3720aba-c7c7-49e0-8119-375a6a086f0d None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:41.273138 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3720aba-c7c7-49e0-8119-375a6a086f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:41.273419 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3720aba-c7c7-49e0-8119-375a6a086f0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:41.326130 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:41.350401 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3720aba-c7c7-49e0-8119-375a6a086f0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:41.351910 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3720aba-c7c7-49e0-8119-375a6a086f0d None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:41.460672 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 223/452] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:12:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 15:12:41.466374 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afabf99-c235-4bbc-ab12-a5914818ae17 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:41.570477 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afabf99-c235-4bbc-ab12-a5914818ae17 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:41.600755 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:41.605562 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82fdf5-c188-4fe7-aba0-f2c35f799fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:41.606209 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82fdf5-c188-4fe7-aba0-f2c35f799fc9 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:41.606494 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82fdf5-c188-4fe7-aba0-f2c35f799fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:41.606773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82fdf5-c188-4fe7-aba0-f2c35f799fc9 None admin] PATH_INFO: `/v3/projects/0744d3ce240a4f198b94f2d1e6d62c8e` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:41.607971 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d82fdf5-c188-4fe7-aba0-f2c35f799fc9 None admin] RBAC: Authorizing `identity:get_project(project_id=0744d3ce240a4f198b94f2d1e6d62c8e)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:41.609439 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d82fdf5-c188-4fe7-aba0-f2c35f799fc9 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:41.611442 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 230/453] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:12:41 2022] GET /identity/v3/projects/0744d3ce240a4f198b94f2d1e6d62c8e => generated 344 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:41.617522 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3720aba-c7c7-49e0-8119-375a6a086f0d None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:41.695712 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:41.717710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3720aba-c7c7-49e0-8119-375a6a086f0d None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:41.752094 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af031d10-4698-4884-a5bf-1f549e628148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:41.752752 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af031d10-4698-4884-a5bf-1f549e628148 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:41.753036 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af031d10-4698-4884-a5bf-1f549e628148 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:41.753312 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af031d10-4698-4884-a5bf-1f549e628148 None admin] PATH_INFO: `/v3/groups/d178fe555c6d4070adcb0068db625fa9` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:41.755063 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af031d10-4698-4884-a5bf-1f549e628148 None admin] RBAC: Authorizing `identity:get_group(group_id=d178fe555c6d4070adcb0068db625fa9)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:41.756590 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af031d10-4698-4884-a5bf-1f549e628148 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:41.758189 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 224/454] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 5 15:12:41 2022] GET /identity/v3/groups/d178fe555c6d4070adcb0068db625fa9 => generated 229 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:41.763975 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82fdf5-c188-4fe7-aba0-f2c35f799fc9 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:41.859567 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82fdf5-c188-4fe7-aba0-f2c35f799fc9 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:41.893432 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a46f5-bb91-44a4-aecd-eddd4568fab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:41.894036 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a46f5-bb91-44a4-aecd-eddd4568fab1 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:41.894323 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a46f5-bb91-44a4-aecd-eddd4568fab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:41.894600 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a46f5-bb91-44a4-aecd-eddd4568fab1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:41.895408 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927a46f5-bb91-44a4-aecd-eddd4568fab1 None admin] RBAC: Adding query filter params (group.id=d178fe555c6d4070adcb0068db625fa9, role.id=7c88c815db854954b516ab97cd471647, scope.project.id=0744d3ce240a4f198b94f2d1e6d62c8e) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:41.895722 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927a46f5-bb91-44a4-aecd-eddd4568fab1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:41.897203 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927a46f5-bb91-44a4-aecd-eddd4568fab1 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:41.902830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 231/455] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Aug 5 15:12:41 2022] GET /identity/v3/role_assignments?group.id=d178fe555c6d4070adcb0068db625fa9&scope.project.id=0744d3ce240a4f198b94f2d1e6d62c8e&role.id=7c88c815db854954b516ab97cd471647 => generated 611 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:44.272920 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29605450-dbfb-4c14-bd64-9c0fd8fe8470 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:44.273972 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29605450-dbfb-4c14-bd64-9c0fd8fe8470 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:44.274739 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29605450-dbfb-4c14-bd64-9c0fd8fe8470 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:44.276648 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 225/456] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:44.282037 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767ac27-651c-403c-954c-7c97eff7afe0 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:44.282858 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767ac27-651c-403c-954c-7c97eff7afe0 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:44.283340 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767ac27-651c-403c-954c-7c97eff7afe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:44.302157 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-5767ac27-651c-403c-954c-7c97eff7afe0 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:44.303685 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-5767ac27-651c-403c-954c-7c97eff7afe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:44.378633 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:44.390559 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 232/457] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:44.437731 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29605450-dbfb-4c14-bd64-9c0fd8fe8470 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:44.572769 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29605450-dbfb-4c14-bd64-9c0fd8fe8470 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:44.619101 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50b864-9949-4b31-8580-bda522f2b48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:44.619762 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50b864-9949-4b31-8580-bda522f2b48f None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:44.620052 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50b864-9949-4b31-8580-bda522f2b48f None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:44.620359 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50b864-9949-4b31-8580-bda522f2b48f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:44.631954 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db50b864-9949-4b31-8580-bda522f2b48f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:44.633460 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db50b864-9949-4b31-8580-bda522f2b48f None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:44.647394 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:44.660298 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-db50b864-9949-4b31-8580-bda522f2b48f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:44.660298 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:44.660298 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:44.660298 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:44.660298 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:44.660298 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:44.660298 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:44.660298 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:44.660298 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:44.660298 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:44.660298 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:44.660298 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:44.660298 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:12:44.660298 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:12:44.660298 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:12:44.661179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:12:44.661179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:44.661179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:44.661179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:44.661179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:44.661179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:44.661179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:44.661179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:44.661179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:44.661179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:44.661179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:44.661179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:44.661179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:44.661179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:44.661179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:44.661179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:44.661179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:44.662045 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:12:44.662045 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:12:44.662045 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:44.662045 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:44.662045 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 226/458] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:12:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:44.673200 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5767ac27-651c-403c-954c-7c97eff7afe0 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:44.803508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9a077-3d10-4e3a-80b7-7d40f56e7651 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:44.803865 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9a077-3d10-4e3a-80b7-7d40f56e7651 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:44.804090 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9a077-3d10-4e3a-80b7-7d40f56e7651 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:44.805703 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 227/459] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 15:12:44.810286 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41c775-32df-45c9-9d24-fa37e5e19793 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:44.810578 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41c775-32df-45c9-9d24-fa37e5e19793 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:44.810863 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41c775-32df-45c9-9d24-fa37e5e19793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:44.813259 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5767ac27-651c-403c-954c-7c97eff7afe0 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:44.837452 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-4f41c775-32df-45c9-9d24-fa37e5e19793 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:44.838771 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-4f41c775-32df-45c9-9d24-fa37e5e19793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:44.852538 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede4efc-8e46-4a32-bbe6-96018b6575b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:44.853544 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede4efc-8e46-4a32-bbe6-96018b6575b5 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:44.854053 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede4efc-8e46-4a32-bbe6-96018b6575b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:44.854601 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede4efc-8e46-4a32-bbe6-96018b6575b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:44.855497 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ede4efc-8e46-4a32-bbe6-96018b6575b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:44.856000 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ede4efc-8e46-4a32-bbe6-96018b6575b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:44.857816 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ede4efc-8e46-4a32-bbe6-96018b6575b5 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:44.870813 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 233/460] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:12:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:44.872316 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8fe83-2410-4771-980e-6dd2b18c9fe7 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:44.872825 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8fe83-2410-4771-980e-6dd2b18c9fe7 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:44.873299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8fe83-2410-4771-980e-6dd2b18c9fe7 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:44.875196 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 234/461] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:44.880484 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad8fe83-2410-4771-980e-6dd2b18c9fe7 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:44.927838 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 228/462] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:44.930076 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6b84f-2b9b-4ea0-8c93-97ce4b563996 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:44.930588 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6b84f-2b9b-4ea0-8c93-97ce4b563996 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:44.931266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6b84f-2b9b-4ea0-8c93-97ce4b563996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:44.941007 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:44.954865 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-0ea6b84f-2b9b-4ea0-8c93-97ce4b563996 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:44.957153 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-0ea6b84f-2b9b-4ea0-8c93-97ce4b563996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:44.993535 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad8fe83-2410-4771-980e-6dd2b18c9fe7 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:45.028881 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a4e569-906f-48ee-be8e-1c050dfc33c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:45.029816 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4e569-906f-48ee-be8e-1c050dfc33c9 None admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:45.030408 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4e569-906f-48ee-be8e-1c050dfc33c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:45.030871 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4e569-906f-48ee-be8e-1c050dfc33c9 None admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:45.031808 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a4e569-906f-48ee-be8e-1c050dfc33c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:45.033486 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a4e569-906f-48ee-be8e-1c050dfc33c9 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:45.044142 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 229/463] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:45.045375 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6b84f-2b9b-4ea0-8c93-97ce4b563996 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:45.112051 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:45.152544 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 235/464] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 5 15:12:44 2022] POST /identity/v3/users => generated 246 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:12:45.153850 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a4e569-906f-48ee-be8e-1c050dfc33c9 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:45.160836 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6b84f-2b9b-4ea0-8c93-97ce4b563996 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:45.210137 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3938bd-a75f-4349-8603-1db102ceae16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:45.210906 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3938bd-a75f-4349-8603-1db102ceae16 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:45.211202 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3938bd-a75f-4349-8603-1db102ceae16 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:45.211444 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3938bd-a75f-4349-8603-1db102ceae16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:45.226501 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3938bd-a75f-4349-8603-1db102ceae16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:45.228753 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3938bd-a75f-4349-8603-1db102ceae16 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:45.251574 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-4f3938bd-a75f-4349-8603-1db102ceae16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:45.251574 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:45.251574 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:45.251574 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:45.251574 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:45.251574 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:45.251574 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:45.251574 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:45.251574 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:45.251574 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:45.251574 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:45.251574 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:45.251574 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:12:45.251574 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:12:45.251574 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:12:45.252508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:12:45.252508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:45.252508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:45.252508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:45.252508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:45.252508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:45.252508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:45.252508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:45.252508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:45.252508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:45.252508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:45.252508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:45.252508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:45.252508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:45.252508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:45.252508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:45.252508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:45.253360 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:12:45.253360 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:12:45.253360 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:45.253360 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:45.256958 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 230/465] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:12:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:45.258884 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73d28b-6c9a-4a9b-a734-4aa92baa63b2 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:45.259529 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73d28b-6c9a-4a9b-a734-4aa92baa63b2 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:45.260919 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73d28b-6c9a-4a9b-a734-4aa92baa63b2 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:45.262931 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 231/466] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:45.263935 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c73d28b-6c9a-4a9b-a734-4aa92baa63b2 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:45.302472 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a4e569-906f-48ee-be8e-1c050dfc33c9 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:45.319000 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:45.351268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22750c7a-4a7b-4dc0-ae3f-e8f0d7b0cbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:45.351922 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22750c7a-4a7b-4dc0-ae3f-e8f0d7b0cbb7 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:45.352265 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22750c7a-4a7b-4dc0-ae3f-e8f0d7b0cbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:45.352795 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22750c7a-4a7b-4dc0-ae3f-e8f0d7b0cbb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:45.369320 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22750c7a-4a7b-4dc0-ae3f-e8f0d7b0cbb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:45.370884 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22750c7a-4a7b-4dc0-ae3f-e8f0d7b0cbb7 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:45.387415 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-22750c7a-4a7b-4dc0-ae3f-e8f0d7b0cbb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:45.387415 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:45.387415 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:45.387415 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:45.387415 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:45.387415 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:45.387415 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:45.387415 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:45.387415 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:45.387415 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:45.387415 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:45.387415 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:45.387415 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:12:45.387415 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:12:45.387415 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:12:45.389273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:12:45.389273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:45.389273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:45.389273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:45.389273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:45.389273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:45.389273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:45.389273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:45.389273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:45.389273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:45.389273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:45.389273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:45.389273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:45.389273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:45.389273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:45.389273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:45.389273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:45.390137 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:12:45.390137 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:12:45.390137 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:45.390137 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:45.390137 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 236/467] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:12:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:45.390416 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19f8a1-83b8-4980-9b9f-7ac88e7122bc None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:45.390493 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19f8a1-83b8-4980-9b9f-7ac88e7122bc None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:45.390751 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19f8a1-83b8-4980-9b9f-7ac88e7122bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:45.406267 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c73d28b-6c9a-4a9b-a734-4aa92baa63b2 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:45.412256 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-4f19f8a1-83b8-4980-9b9f-7ac88e7122bc None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:45.413452 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-4f19f8a1-83b8-4980-9b9f-7ac88e7122bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:45.418702 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:45.455088 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccba2a6-01f4-4225-ba00-9eec7d9a68de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:45.456093 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccba2a6-01f4-4225-ba00-9eec7d9a68de None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:45.456596 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccba2a6-01f4-4225-ba00-9eec7d9a68de None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:45.457529 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccba2a6-01f4-4225-ba00-9eec7d9a68de None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:45.458426 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccba2a6-01f4-4225-ba00-9eec7d9a68de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:45.458977 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccba2a6-01f4-4225-ba00-9eec7d9a68de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:45.461396 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccba2a6-01f4-4225-ba00-9eec7d9a68de None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:45.480891 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 232/468] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:12:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:45.481755 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccba2a6-01f4-4225-ba00-9eec7d9a68de None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:45.489665 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 237/469] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:45.490532 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f19f8a1-83b8-4980-9b9f-7ac88e7122bc None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:45.602898 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:45.612137 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f19f8a1-83b8-4980-9b9f-7ac88e7122bc None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:45.631736 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccba2a6-01f4-4225-ba00-9eec7d9a68de None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:45.658600 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40278bd-0844-4b79-babe-fdf4b535d325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:45.660671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40278bd-0844-4b79-babe-fdf4b535d325 None admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:45.661137 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40278bd-0844-4b79-babe-fdf4b535d325 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:45.661758 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40278bd-0844-4b79-babe-fdf4b535d325 None admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:45.662961 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40278bd-0844-4b79-babe-fdf4b535d325 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:45.665091 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40278bd-0844-4b79-babe-fdf4b535d325 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:45.667412 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53150208-f012-43a7-a1df-7d8df360bb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:45.668121 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53150208-f012-43a7-a1df-7d8df360bb1f None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:45.668473 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53150208-f012-43a7-a1df-7d8df360bb1f None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:45.668820 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53150208-f012-43a7-a1df-7d8df360bb1f None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:45.669585 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53150208-f012-43a7-a1df-7d8df360bb1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:45.669958 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53150208-f012-43a7-a1df-7d8df360bb1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:45.671377 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53150208-f012-43a7-a1df-7d8df360bb1f None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:45.683982 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 233/470] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:12:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:45.688831 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53150208-f012-43a7-a1df-7d8df360bb1f None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:45.719240 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:45.742788 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 238/471] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 5 15:12:45 2022] POST /identity/v3/users => generated 248 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:12:45.743890 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40278bd-0844-4b79-babe-fdf4b535d325 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:45.823699 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53150208-f012-43a7-a1df-7d8df360bb1f None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:45.874147 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2ed360-e2b6-4cfe-8dd9-b5cbf7118f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:45.874803 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ed360-e2b6-4cfe-8dd9-b5cbf7118f9e None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:45.875085 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ed360-e2b6-4cfe-8dd9-b5cbf7118f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:45.875362 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ed360-e2b6-4cfe-8dd9-b5cbf7118f9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:45.887763 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2ed360-e2b6-4cfe-8dd9-b5cbf7118f9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:45.889209 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2ed360-e2b6-4cfe-8dd9-b5cbf7118f9e None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:45.905746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-0e2ed360-e2b6-4cfe-8dd9-b5cbf7118f9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:45.905746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:45.905746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:45.905746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:45.905746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:45.905746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:45.905746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:45.905746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:45.905746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:45.905746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:45.905746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:45.905746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:45.905746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:12:45.905746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:12:45.905746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:12:45.906833 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:12:45.906833 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:45.906833 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:45.906833 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:45.906833 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:45.906833 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:45.906833 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:45.906833 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:45.906833 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:45.906833 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:45.906833 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:45.906833 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:45.906833 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:45.906833 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:45.906833 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:45.906833 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:45.906833 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:45.907791 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:12:45.907791 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:12:45.907791 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:45.907791 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:45.907791 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 234/472] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:12:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:45.908058 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd85070-ec22-424a-beac-b5cdfbaf6291 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:45.908715 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40278bd-0844-4b79-babe-fdf4b535d325 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:45.909169 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd85070-ec22-424a-beac-b5cdfbaf6291 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:45.909453 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd85070-ec22-424a-beac-b5cdfbaf6291 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:45.915314 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 235/473] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:45.917650 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd85070-ec22-424a-beac-b5cdfbaf6291 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:45.955488 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e02ac5-9440-4d62-adff-14a87d1e89b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:45.956149 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e02ac5-9440-4d62-adff-14a87d1e89b5 None admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:45.956435 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e02ac5-9440-4d62-adff-14a87d1e89b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:45.956763 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e02ac5-9440-4d62-adff-14a87d1e89b5 None admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:45.957475 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e02ac5-9440-4d62-adff-14a87d1e89b5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:45.958923 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e02ac5-9440-4d62-adff-14a87d1e89b5 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:46.017097 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:46.037261 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd85070-ec22-424a-beac-b5cdfbaf6291 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:46.053556 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 239/474] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 5 15:12:45 2022] POST /identity/v3/users => generated 248 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:12:46.055619 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a565ab-428e-4638-b4a0-0e14bd027608 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:46.056427 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a565ab-428e-4638-b4a0-0e14bd027608 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:46.056998 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a565ab-428e-4638-b4a0-0e14bd027608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:46.074444 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82e30f0-52bd-4758-bb56-afa305b09d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:46.075092 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e30f0-52bd-4758-bb56-afa305b09d3e None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:46.075347 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e30f0-52bd-4758-bb56-afa305b09d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:46.075677 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e30f0-52bd-4758-bb56-afa305b09d3e None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:46.076417 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82e30f0-52bd-4758-bb56-afa305b09d3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:46.076725 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82e30f0-52bd-4758-bb56-afa305b09d3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:46.078338 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82e30f0-52bd-4758-bb56-afa305b09d3e None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:46.080360 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:46.083677 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-58a565ab-428e-4638-b4a0-0e14bd027608 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:46.085345 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-58a565ab-428e-4638-b4a0-0e14bd027608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:46.092852 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 236/475] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:12:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:46.099049 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82e30f0-52bd-4758-bb56-afa305b09d3e None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:46.211537 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 240/476] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:46.259562 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82e30f0-52bd-4758-bb56-afa305b09d3e None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:46.262976 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a565ab-428e-4638-b4a0-0e14bd027608 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:46.318884 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98097e91-92e7-4e27-9215-35910357d267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:46.319508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98097e91-92e7-4e27-9215-35910357d267 None admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:46.319791 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98097e91-92e7-4e27-9215-35910357d267 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:46.320127 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98097e91-92e7-4e27-9215-35910357d267 None admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:46.320786 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98097e91-92e7-4e27-9215-35910357d267 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:46.322226 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98097e91-92e7-4e27-9215-35910357d267 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:46.383842 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 237/477] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 5 15:12:46 2022] POST /identity/v3/users => generated 249 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:12:46.384546 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:46.408483 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a565ab-428e-4638-b4a0-0e14bd027608 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:46.456327 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123b4d36-b201-47cd-9a29-4e25568e9aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:46.457498 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b4d36-b201-47cd-9a29-4e25568e9aff None admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:46.457788 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b4d36-b201-47cd-9a29-4e25568e9aff None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:46.458069 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b4d36-b201-47cd-9a29-4e25568e9aff None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:46.458833 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123b4d36-b201-47cd-9a29-4e25568e9aff None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:46.460254 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123b4d36-b201-47cd-9a29-4e25568e9aff None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:46.464109 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:46.518974 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.sql.core [None req-123b4d36-b201-47cd-9a29-4e25568e9aff None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 05 15:12:46.518974 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 05 15:12:46.518974 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [parameters: {'id': '3af1be207cb44dd2b6cd3fc2a9f5e6b4', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 05 15:12:46.518974 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80541) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 05 15:12:46.539211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-123b4d36-b201-47cd-9a29-4e25568e9aff None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 05 15:12:46.539211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:46.539211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 15:12:46.539211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 15:12:46.539211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 15:12:46.539211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 15:12:46.539211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 15:12:46.539211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 15:12:46.539211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 15:12:46.539211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application conn.query(q) Aug 05 15:12:46.539211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 15:12:46.539211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 15:12:46.539211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 15:12:46.539211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application result.read() Aug 05 15:12:46.539211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 15:12:46.540299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 15:12:46.540299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 15:12:46.540299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 15:12:46.540299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 15:12:46.540299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 15:12:46.540299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 15:12:46.540299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 15:12:46.540299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 05 15:12:46.540299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:46.540299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 05 15:12:46.540299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:46.540299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:46.540299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 05 15:12:46.540299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 05 15:12:46.540299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 05 15:12:46.540299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return ref.to_dict() Aug 05 15:12:46.540299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 15:12:46.540299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application next(self.gen) Aug 05 15:12:46.541664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 05 15:12:46.541664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application yield resource Aug 05 15:12:46.541664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 15:12:46.541664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application next(self.gen) Aug 05 15:12:46.541664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 05 15:12:46.541664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self.session.rollback() Aug 05 15:12:46.541664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 05 15:12:46.541664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self.force_reraise() Aug 05 15:12:46.541664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 05 15:12:46.541664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise self.value Aug 05 15:12:46.541664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 05 15:12:46.541664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 05 15:12:46.541664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 05 15:12:46.541664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application session.commit() Aug 05 15:12:46.541664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 05 15:12:46.541664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 05 15:12:46.541664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 05 15:12:46.542548 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self._prepare_impl() Aug 05 15:12:46.542548 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 05 15:12:46.542548 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self.session.flush() Aug 05 15:12:46.542548 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 05 15:12:46.542548 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self._flush(objects) Aug 05 15:12:46.542548 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 05 15:12:46.542548 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 05 15:12:46.542548 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 05 15:12:46.542548 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application compat.raise_( Aug 05 15:12:46.542548 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 15:12:46.542548 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception Aug 05 15:12:46.542548 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 05 15:12:46.542548 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application flush_context.execute() Aug 05 15:12:46.542548 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 05 15:12:46.542548 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rec.execute(self) Aug 05 15:12:46.542548 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 05 15:12:46.542548 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 05 15:12:46.543410 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 05 15:12:46.543410 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 05 15:12:46.543410 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 05 15:12:46.543410 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 05 15:12:46.543410 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 05 15:12:46.543410 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 05 15:12:46.543410 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 05 15:12:46.543410 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 05 15:12:46.543410 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 05 15:12:46.543410 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 05 15:12:46.543410 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 05 15:12:46.543410 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 05 15:12:46.543410 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 05 15:12:46.543410 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 05 15:12:46.543410 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 15:12:46.543410 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception Aug 05 15:12:46.545211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 15:12:46.545211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 15:12:46.545211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 15:12:46.545211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 15:12:46.545211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 15:12:46.545211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 15:12:46.545211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 15:12:46.545211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application conn.query(q) Aug 05 15:12:46.545211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 15:12:46.545211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 15:12:46.545211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 15:12:46.545211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application result.read() Aug 05 15:12:46.545211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 15:12:46.545211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 15:12:46.545211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 15:12:46.545211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 15:12:46.547227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 15:12:46.547227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 15:12:46.547227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 15:12:46.547227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 15:12:46.547227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 05 15:12:46.547227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 05 15:12:46.547227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [parameters: {'id': '3af1be207cb44dd2b6cd3fc2a9f5e6b4', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 05 15:12:46.547227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 05 15:12:46.547227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:46.547227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 15:12:46.547227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:46.547227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:46.547227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:46.547227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:46.547227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:46.547227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:46.552958 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:46.552958 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:46.552958 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:46.552958 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:46.552958 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:46.552958 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:46.552958 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 05 15:12:46.552958 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 05 15:12:46.552958 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:46.552958 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:46.552958 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 05 15:12:46.552958 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 05 15:12:46.552958 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 05 15:12:46.552958 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 05 15:12:46.552958 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 05 15:12:46.552958 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:46.556425 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 241/478] 10.222.0.40 () {60 vars in 1131 bytes} [Fri Aug 5 15:12:46 2022] POST /identity/v3/roles => generated 144 bytes in 278 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:46.556575 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98097e91-92e7-4e27-9215-35910357d267 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:46.676866 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98097e91-92e7-4e27-9215-35910357d267 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:46.711983 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f79fae7-928d-49b4-bd41-ada6ab3eba9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:46.712580 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79fae7-928d-49b4-bd41-ada6ab3eba9f None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:46.712860 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79fae7-928d-49b4-bd41-ada6ab3eba9f None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:46.713134 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79fae7-928d-49b4-bd41-ada6ab3eba9f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:46.723534 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f79fae7-928d-49b4-bd41-ada6ab3eba9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:46.729023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-9f79fae7-928d-49b4-bd41-ada6ab3eba9f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 15:12:46.729023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:46.729023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:46.729023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:46.729023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:46.729023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:46.729023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:46.729023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:46.729023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:46.729023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:46.729023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:46.729023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:46.729023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:46.729023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:46.729023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:46.729023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:46.731204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:46.731204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:46.731204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:46.731204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:46.731204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:46.731204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:46.731204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:46.731204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:46.731204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:46.731204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:46.731204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:46.731204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:46.731204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:46.731204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:46.731204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:46.731204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:46.733252 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:46.733252 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:46.733252 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:46.733252 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:46.733252 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:46.733252 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:46.733252 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:46.733252 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:46.733252 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:46.733252 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:46.733252 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 15:12:46.733252 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:46.733252 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 238/479] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 15:12:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:46.740192 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123b4d36-b201-47cd-9a29-4e25568e9aff None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:46.853959 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123b4d36-b201-47cd-9a29-4e25568e9aff None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:46.893738 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda539f3-ca09-4b9f-b394-210b69d4380c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:46.894409 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda539f3-ca09-4b9f-b394-210b69d4380c None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:46.894699 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda539f3-ca09-4b9f-b394-210b69d4380c None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:46.894983 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda539f3-ca09-4b9f-b394-210b69d4380c None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:46.895708 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda539f3-ca09-4b9f-b394-210b69d4380c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:46.896024 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda539f3-ca09-4b9f-b394-210b69d4380c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:46.897436 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda539f3-ca09-4b9f-b394-210b69d4380c None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:46.908970 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 242/480] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:12:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:47.490861 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ebf53-b307-4cd3-be63-f393c02d2a61 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:47.491211 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ebf53-b307-4cd3-be63-f393c02d2a61 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:47.491399 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ebf53-b307-4cd3-be63-f393c02d2a61 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:47.493014 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 239/481] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:47.499263 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88baf5-b994-402b-b74d-18fc387e7f2f None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:47.499837 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88baf5-b994-402b-b74d-18fc387e7f2f None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:47.499837 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88baf5-b994-402b-b74d-18fc387e7f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:47.524276 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-0c88baf5-b994-402b-b74d-18fc387e7f2f None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:47.525485 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-0c88baf5-b994-402b-b74d-18fc387e7f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:47.610076 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:47.626693 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 243/482] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 15:12:47.667923 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ebf53-b307-4cd3-be63-f393c02d2a61 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:47.751070 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:47.795231 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ebf53-b307-4cd3-be63-f393c02d2a61 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:47.829268 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf5308-2d0f-4494-a912-c7f0a2920204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:47.829919 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf5308-2d0f-4494-a912-c7f0a2920204 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:47.830204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf5308-2d0f-4494-a912-c7f0a2920204 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:47.830500 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf5308-2d0f-4494-a912-c7f0a2920204 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:47.841627 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcf5308-2d0f-4494-a912-c7f0a2920204 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:47.847180 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-efcf5308-2d0f-4494-a912-c7f0a2920204 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:12:47.847180 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:47.847180 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:47.847180 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:47.847180 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:47.847180 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:47.847180 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:47.847180 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:47.847180 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:47.847180 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:47.847180 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:47.847180 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:47.847180 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:47.847180 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:47.847180 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:47.847180 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:47.848662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:47.848662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:47.848662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:47.848662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:47.848662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:47.848662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:47.848662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:47.848662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:47.848662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:47.848662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:47.848662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:47.848662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:47.848662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:47.848662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:47.848662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:47.848662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:47.849458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:47.849458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:47.849458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:47.849458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:47.849458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:47.849458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:47.849458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:47.849458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:47.849458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:47.849458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:47.849458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:12:47.849458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:47.849458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 240/483] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 15:12:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:47.854763 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88baf5-b994-402b-b74d-18fc387e7f2f None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:48.004483 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88baf5-b994-402b-b74d-18fc387e7f2f None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:48.056872 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c649c9c-7f9a-4b23-a2eb-99efef407a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:48.057115 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea437a-ac82-44b3-9e78-d112a3b0ad65 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:48.057415 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea437a-ac82-44b3-9e78-d112a3b0ad65 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:48.057699 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea437a-ac82-44b3-9e78-d112a3b0ad65 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:48.058204 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c649c9c-7f9a-4b23-a2eb-99efef407a98 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:48.058673 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c649c9c-7f9a-4b23-a2eb-99efef407a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:48.059085 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c649c9c-7f9a-4b23-a2eb-99efef407a98 None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:48.059269 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 241/484] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:48.060115 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c649c9c-7f9a-4b23-a2eb-99efef407a98 None admin] RBAC: Adding query filter params (name=service) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:48.060566 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c649c9c-7f9a-4b23-a2eb-99efef407a98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:48.062075 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c649c9c-7f9a-4b23-a2eb-99efef407a98 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:48.067963 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607e4db-adac-4783-89ea-017a1e106523 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:48.067963 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607e4db-adac-4783-89ea-017a1e106523 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:48.067963 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607e4db-adac-4783-89ea-017a1e106523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:48.079053 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 244/485] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 15:12:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:48.089049 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c649c9c-7f9a-4b23-a2eb-99efef407a98 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:48.094240 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-c607e4db-adac-4783-89ea-017a1e106523 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:48.095441 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-c607e4db-adac-4783-89ea-017a1e106523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:48.187187 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:48.196917 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 242/486] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 15:12:48.261861 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c607e4db-adac-4783-89ea-017a1e106523 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:48.262701 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c649c9c-7f9a-4b23-a2eb-99efef407a98 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:48.309855 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad81fa-5fdd-4297-b48e-e34010f04388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:48.310507 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad81fa-5fdd-4297-b48e-e34010f04388 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:48.310796 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad81fa-5fdd-4297-b48e-e34010f04388 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:48.311079 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad81fa-5fdd-4297-b48e-e34010f04388 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:48.334704 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ad81fa-5fdd-4297-b48e-e34010f04388 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:48.336252 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ad81fa-5fdd-4297-b48e-e34010f04388 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:48.340852 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:48.365173 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-23ad81fa-5fdd-4297-b48e-e34010f04388 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:48.365173 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:48.365173 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:48.365173 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:48.365173 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:48.365173 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:48.365173 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:48.365173 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:48.365173 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:48.365173 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:48.365173 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:48.365173 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:48.365173 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:12:48.365173 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:12:48.365173 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:12:48.366071 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:12:48.366071 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:48.366071 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:48.366071 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:48.366071 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:48.366071 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:48.366071 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:48.366071 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:48.366071 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:48.366071 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:48.366071 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:48.366071 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:48.366071 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:48.366071 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:48.366071 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:48.366071 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:48.366071 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:48.366983 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:12:48.366983 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:12:48.366983 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:48.366983 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:48.366983 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 245/487] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:12:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:48.373575 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad81fa-5fdd-4297-b48e-e34010f04388 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:48.449517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c607e4db-adac-4783-89ea-017a1e106523 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:48.517691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71d9acc-caad-4bd5-855e-da7efaa85cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:48.518265 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d9acc-caad-4bd5-855e-da7efaa85cca None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:48.518552 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d9acc-caad-4bd5-855e-da7efaa85cca None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:48.518831 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d9acc-caad-4bd5-855e-da7efaa85cca None admin] PATH_INFO: `/v3/roles/service` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:48.532703 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71d9acc-caad-4bd5-855e-da7efaa85cca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:48.538269 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-b71d9acc-caad-4bd5-855e-da7efaa85cca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:12:48.538269 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:48.538269 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:48.538269 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:48.538269 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:48.538269 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:48.538269 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:48.538269 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:48.538269 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:48.538269 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:48.538269 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:48.538269 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:48.538269 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:48.538269 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:48.538269 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:48.538269 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:48.540364 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:48.540364 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:48.540364 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:48.540364 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:48.540364 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:48.540364 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:48.540364 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:48.540364 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:48.540364 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:48.540364 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:48.540364 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:48.540364 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:48.540364 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:48.540364 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:48.540364 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:48.540364 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:48.542408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:48.542408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:48.542408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:48.542408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:48.542408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:48.542408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:48.542408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:48.542408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:48.542408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:48.542408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:48.542408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:12:48.542408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:48.542408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 243/488] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 15:12:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:12:48.543611 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad81fa-5fdd-4297-b48e-e34010f04388 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:48.547668 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:48.547774 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10b3ba-2af5-4ccf-a1f8-39071f86fb12 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:48.547774 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10b3ba-2af5-4ccf-a1f8-39071f86fb12 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:48.547774 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10b3ba-2af5-4ccf-a1f8-39071f86fb12 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:48.549919 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 244/489] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:48.549919 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad10b3ba-2af5-4ccf-a1f8-39071f86fb12 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:48.607976 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7966f1d6-ba83-460c-b046-c60968c778c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:48.608950 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966f1d6-ba83-460c-b046-c60968c778c8 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:48.609378 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966f1d6-ba83-460c-b046-c60968c778c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:48.609786 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966f1d6-ba83-460c-b046-c60968c778c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:48.610487 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7966f1d6-ba83-460c-b046-c60968c778c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:48.610798 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7966f1d6-ba83-460c-b046-c60968c778c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:48.612195 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7966f1d6-ba83-460c-b046-c60968c778c8 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:48.629621 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 246/490] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:12:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:12:48.630933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c2a58-981a-467f-996d-8baa5ef7d042 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:48.631281 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c2a58-981a-467f-996d-8baa5ef7d042 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:48.631575 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c2a58-981a-467f-996d-8baa5ef7d042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:48.649881 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-0a4c2a58-981a-467f-996d-8baa5ef7d042 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:48.651090 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-0a4c2a58-981a-467f-996d-8baa5ef7d042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:48.707875 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad10b3ba-2af5-4ccf-a1f8-39071f86fb12 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:48.720507 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:48.732086 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 247/491] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:48.732086 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4c2a58-981a-467f-996d-8baa5ef7d042 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:48.766846 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f36c36-a525-446b-bb19-f878d9591710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:48.768245 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f36c36-a525-446b-bb19-f878d9591710 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:48.768772 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f36c36-a525-446b-bb19-f878d9591710 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:48.769426 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f36c36-a525-446b-bb19-f878d9591710 None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:48.770711 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f36c36-a525-446b-bb19-f878d9591710 None admin] RBAC: Adding query filter params (name=service) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:48.771521 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f36c36-a525-446b-bb19-f878d9591710 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:48.773657 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f36c36-a525-446b-bb19-f878d9591710 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:48.785421 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 245/492] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 15:12:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:48.786548 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f36c36-a525-446b-bb19-f878d9591710 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:48.859418 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:48.869678 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4c2a58-981a-467f-996d-8baa5ef7d042 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:48.913627 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f36c36-a525-446b-bb19-f878d9591710 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:48.914541 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09debf52-89ef-42d2-bbf6-be8922eeabde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:48.915192 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09debf52-89ef-42d2-bbf6-be8922eeabde None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:48.915479 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09debf52-89ef-42d2-bbf6-be8922eeabde None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:48.915760 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09debf52-89ef-42d2-bbf6-be8922eeabde None admin] PATH_INFO: `/v3/users/nova` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:48.926231 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09debf52-89ef-42d2-bbf6-be8922eeabde None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:48.928142 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09debf52-89ef-42d2-bbf6-be8922eeabde None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:48.946662 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-09debf52-89ef-42d2-bbf6-be8922eeabde None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 15:12:48.946662 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:48.946662 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:48.946662 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:48.946662 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:48.946662 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:48.946662 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:48.946662 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:48.946662 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:48.946662 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:48.946662 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:48.946662 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:48.946662 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 15:12:48.946662 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 15:12:48.946662 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 15:12:48.946662 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 15:12:48.947610 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:48.947610 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:48.947610 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 15:12:48.947610 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:12:48.947610 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 15:12:48.947610 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:12:48.947610 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:48.947610 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:48.947610 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:48.947610 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:48.947610 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:48.947610 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:48.947610 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:48.947610 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:48.947610 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:48.947610 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:48.947610 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:48.947610 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:48.948553 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:48.948553 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:48.948553 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 15:12:48.948553 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 15:12:48.948553 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 15:12:48.948553 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 15:12:48.948553 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 15:12:48.948553 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 15:12:48.948553 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 15:12:48.948553 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:48.950858 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 248/493] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Aug 5 15:12:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:48.951837 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09debf52-89ef-42d2-bbf6-be8922eeabde None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:48.964049 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39246f8-9481-4b56-8c7e-ecf6e8947caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:48.964727 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39246f8-9481-4b56-8c7e-ecf6e8947caa None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:48.965014 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39246f8-9481-4b56-8c7e-ecf6e8947caa None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:48.965293 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39246f8-9481-4b56-8c7e-ecf6e8947caa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:48.977300 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39246f8-9481-4b56-8c7e-ecf6e8947caa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:48.982880 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-d39246f8-9481-4b56-8c7e-ecf6e8947caa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:12:48.982880 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:48.982880 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:48.982880 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:48.982880 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:48.982880 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:48.982880 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:48.982880 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:48.982880 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:48.982880 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:48.982880 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:48.982880 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:48.982880 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:48.982880 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:48.982880 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:48.982880 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:48.983764 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:48.983764 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:48.983764 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:48.983764 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:48.983764 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:48.983764 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:48.983764 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:48.983764 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:48.983764 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:48.983764 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:48.983764 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:48.983764 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:48.983764 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:48.983764 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:48.983764 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:48.983764 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:48.984613 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:48.984613 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:48.984613 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:48.984613 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:48.984613 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:48.984613 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:48.984613 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:48.984613 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:48.984613 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:48.984613 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:48.984613 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:12:48.984613 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:48.989538 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 246/494] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 15:12:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:48.990377 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39246f8-9481-4b56-8c7e-ecf6e8947caa None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:49.024759 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:49.063734 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09debf52-89ef-42d2-bbf6-be8922eeabde None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:49.100046 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d152bbae-3666-4d4f-a062-86928b079857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:49.101050 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152bbae-3666-4d4f-a062-86928b079857 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:49.101618 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152bbae-3666-4d4f-a062-86928b079857 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:49.102117 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152bbae-3666-4d4f-a062-86928b079857 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:49.103040 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39246f8-9481-4b56-8c7e-ecf6e8947caa None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:49.115285 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d152bbae-3666-4d4f-a062-86928b079857 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:49.117126 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d152bbae-3666-4d4f-a062-86928b079857 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:49.133103 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-d152bbae-3666-4d4f-a062-86928b079857 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:49.133103 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:49.133103 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:49.133103 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:49.133103 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:49.133103 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:49.133103 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:49.133103 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:49.133103 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:49.133103 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:49.133103 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:49.133103 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:49.133103 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:12:49.133103 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:12:49.133103 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:12:49.133969 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:12:49.133969 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:49.133969 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:49.133969 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:49.133969 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:49.133969 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:49.133969 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:49.133969 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:49.133969 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:49.133969 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:49.133969 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:49.133969 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:49.133969 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:49.133969 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:49.133969 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:49.133969 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:49.133969 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:49.134827 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:12:49.134827 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:12:49.134827 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:49.134827 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:49.136707 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 249/495] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:12:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:49.137704 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d152bbae-3666-4d4f-a062-86928b079857 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:49.146452 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f8610b-9435-4e41-8f65-5375a7a1ac78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:49.147085 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8610b-9435-4e41-8f65-5375a7a1ac78 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:49.147375 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8610b-9435-4e41-8f65-5375a7a1ac78 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:49.147657 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8610b-9435-4e41-8f65-5375a7a1ac78 None admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:49.148503 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f8610b-9435-4e41-8f65-5375a7a1ac78 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:49.148819 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f8610b-9435-4e41-8f65-5375a7a1ac78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:49.149970 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:49.151431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f8610b-9435-4e41-8f65-5375a7a1ac78 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:49.194121 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 247/496] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Aug 5 15:12:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:49.195365 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f8610b-9435-4e41-8f65-5375a7a1ac78 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:49.250082 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d152bbae-3666-4d4f-a062-86928b079857 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:49.286396 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9e9e9f-c5d0-4a3f-9a1c-e22c9e0ad843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:49.286999 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e9e9f-c5d0-4a3f-9a1c-e22c9e0ad843 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:49.287281 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e9e9f-c5d0-4a3f-9a1c-e22c9e0ad843 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:49.287553 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e9e9f-c5d0-4a3f-9a1c-e22c9e0ad843 None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:49.288347 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9e9e9f-c5d0-4a3f-9a1c-e22c9e0ad843 None admin] RBAC: Adding query filter params (name=service) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:49.288652 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9e9e9f-c5d0-4a3f-9a1c-e22c9e0ad843 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:49.290006 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9e9e9f-c5d0-4a3f-9a1c-e22c9e0ad843 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:49.292934 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:49.303584 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 250/497] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 15:12:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:49.303584 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9e9e9f-c5d0-4a3f-9a1c-e22c9e0ad843 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:49.310249 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f8610b-9435-4e41-8f65-5375a7a1ac78 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:49.350849 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384e2415-8c08-41f5-8248-866644a59fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:49.351472 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e2415-8c08-41f5-8248-866644a59fdf None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:49.351759 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e2415-8c08-41f5-8248-866644a59fdf None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:49.352039 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e2415-8c08-41f5-8248-866644a59fdf None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:49.352766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384e2415-8c08-41f5-8248-866644a59fdf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:49.353239 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384e2415-8c08-41f5-8248-866644a59fdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:49.354762 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384e2415-8c08-41f5-8248-866644a59fdf None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:49.367137 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:49.373360 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 248/498] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:12:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:49.373360 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281a531-6246-4e72-9ebd-6709d3876af2 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:49.373360 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281a531-6246-4e72-9ebd-6709d3876af2 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:49.373360 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281a531-6246-4e72-9ebd-6709d3876af2 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:49.373360 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 249/499] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:49.373360 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b281a531-6246-4e72-9ebd-6709d3876af2 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:49.409533 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9e9e9f-c5d0-4a3f-9a1c-e22c9e0ad843 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:49.445910 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd723ef-ea94-402a-a02d-5993c911b0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:49.446521 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd723ef-ea94-402a-a02d-5993c911b0a6 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:49.446806 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd723ef-ea94-402a-a02d-5993c911b0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:49.447085 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd723ef-ea94-402a-a02d-5993c911b0a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:49.489632 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b281a531-6246-4e72-9ebd-6709d3876af2 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:49.516122 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd723ef-ea94-402a-a02d-5993c911b0a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:49.517669 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd723ef-ea94-402a-a02d-5993c911b0a6 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:49.526388 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d511e2e-fc77-4b99-acc6-966c73b9e69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:49.527383 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d511e2e-fc77-4b99-acc6-966c73b9e69d None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:49.527670 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d511e2e-fc77-4b99-acc6-966c73b9e69d None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:49.527939 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d511e2e-fc77-4b99-acc6-966c73b9e69d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:49.540167 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d511e2e-fc77-4b99-acc6-966c73b9e69d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:49.541728 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d511e2e-fc77-4b99-acc6-966c73b9e69d None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:49.550962 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:49.558128 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-2d511e2e-fc77-4b99-acc6-966c73b9e69d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:49.558128 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:49.558128 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:49.558128 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:49.558128 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:49.558128 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:49.558128 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:49.558128 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:49.558128 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:49.558128 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:49.558128 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:49.558128 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:49.558128 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:12:49.558128 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:12:49.558128 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:12:49.563015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:12:49.563015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:49.563015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:49.563015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:49.563015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:49.563015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:49.563015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:49.563015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:49.563015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:49.563015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:49.563015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:49.563015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:49.563015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:49.563015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:49.563015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:49.563015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:49.563015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:49.564247 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:12:49.564247 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:12:49.564247 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:49.564247 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:49.564247 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 250/500] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:12:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:49.564247 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d511e2e-fc77-4b99-acc6-966c73b9e69d None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:49.590728 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:49.626572 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 251/501] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:12:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 15:12:49.627810 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7295d1b-52d9-4b9e-9d8b-7e15d6af569f None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:49.628116 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7295d1b-52d9-4b9e-9d8b-7e15d6af569f None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:49.628417 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7295d1b-52d9-4b9e-9d8b-7e15d6af569f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:49.645527 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-e7295d1b-52d9-4b9e-9d8b-7e15d6af569f None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:49.646739 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-e7295d1b-52d9-4b9e-9d8b-7e15d6af569f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:49.664085 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d511e2e-fc77-4b99-acc6-966c73b9e69d None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:49.697211 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440b8172-e3ab-453a-9291-554a7022a4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:49.697840 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b8172-e3ab-453a-9291-554a7022a4d6 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:49.698131 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b8172-e3ab-453a-9291-554a7022a4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:49.698409 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b8172-e3ab-453a-9291-554a7022a4d6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:49.705669 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440b8172-e3ab-453a-9291-554a7022a4d6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:49.707120 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440b8172-e3ab-453a-9291-554a7022a4d6 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:49.713085 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 252/502] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:49.713898 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7295d1b-52d9-4b9e-9d8b-7e15d6af569f None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:49.719757 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-440b8172-e3ab-453a-9291-554a7022a4d6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 05 15:12:49.719757 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:49.719757 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:49.719757 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:49.719757 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:49.719757 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:49.719757 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:49.719757 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:49.719757 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:49.719757 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:49.719757 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:49.719757 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:49.719757 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 15:12:49.719757 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 15:12:49.719757 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 15:12:49.719757 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 15:12:49.721126 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:49.721126 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:49.721126 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 15:12:49.721126 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:12:49.721126 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 15:12:49.721126 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:12:49.721126 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:49.721126 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:49.721126 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:49.721126 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:49.721126 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:49.721126 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:49.721126 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:49.721126 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:49.721126 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:49.721126 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:49.721126 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:49.721126 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:49.722026 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:49.722026 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:49.722026 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 15:12:49.722026 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 15:12:49.722026 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 15:12:49.722026 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 15:12:49.722026 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 15:12:49.722026 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 15:12:49.722026 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 05 15:12:49.722026 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:49.722026 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 251/503] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:12:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:49.723098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8e38f-5996-40bf-9f8f-aa4ad46289f9 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:49.723098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8e38f-5996-40bf-9f8f-aa4ad46289f9 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:49.723225 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8e38f-5996-40bf-9f8f-aa4ad46289f9 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:49.728428 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 252/504] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:49.728428 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd8e38f-5996-40bf-9f8f-aa4ad46289f9 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:49.795350 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:49.820904 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:49.824598 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7295d1b-52d9-4b9e-9d8b-7e15d6af569f None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:49.837096 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd8e38f-5996-40bf-9f8f-aa4ad46289f9 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:49.858846 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9491f6b8-923e-4e87-b283-ef87fa5a178a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:49.859458 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491f6b8-923e-4e87-b283-ef87fa5a178a None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:49.859737 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491f6b8-923e-4e87-b283-ef87fa5a178a None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:49.860054 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491f6b8-923e-4e87-b283-ef87fa5a178a None admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:49.860781 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9491f6b8-923e-4e87-b283-ef87fa5a178a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:49.861292 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9491f6b8-923e-4e87-b283-ef87fa5a178a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:49.863017 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9491f6b8-923e-4e87-b283-ef87fa5a178a None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:49.870905 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba045b52-1a13-482c-ad07-70459d821f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:49.871541 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba045b52-1a13-482c-ad07-70459d821f58 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:49.871828 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba045b52-1a13-482c-ad07-70459d821f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:49.872138 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba045b52-1a13-482c-ad07-70459d821f58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:49.874606 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 253/505] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:12:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:49.875370 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9491f6b8-923e-4e87-b283-ef87fa5a178a None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:49.883212 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba045b52-1a13-482c-ad07-70459d821f58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:49.884737 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba045b52-1a13-482c-ad07-70459d821f58 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:49.899404 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-ba045b52-1a13-482c-ad07-70459d821f58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:49.899404 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:49.899404 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:49.899404 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:49.899404 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:49.899404 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:49.899404 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:49.899404 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:49.899404 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:49.899404 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:49.899404 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:49.899404 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:49.899404 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:12:49.899404 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:12:49.899404 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:12:49.900424 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:12:49.900424 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:49.900424 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:49.900424 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:49.900424 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:49.900424 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:49.900424 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:49.900424 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:49.900424 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:49.900424 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:49.900424 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:49.900424 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:49.900424 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:49.900424 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:49.900424 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:49.900424 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:49.900424 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:49.901401 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:12:49.901401 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:12:49.901401 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:49.901401 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:49.901401 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 253/506] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:12:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:49.901772 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a94008-29e8-4944-99a7-2464d0089134 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:49.902042 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a94008-29e8-4944-99a7-2464d0089134 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:49.902325 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a94008-29e8-4944-99a7-2464d0089134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:49.920891 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-54a94008-29e8-4944-99a7-2464d0089134 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:49.922076 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-54a94008-29e8-4944-99a7-2464d0089134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:49.979477 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9491f6b8-923e-4e87-b283-ef87fa5a178a None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:49.989288 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 254/507] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:49.990219 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a94008-29e8-4944-99a7-2464d0089134 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:50.012276 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cb148e-85e7-40d9-bbc5-21931b7b50cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:50.012881 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cb148e-85e7-40d9-bbc5-21931b7b50cf None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:50.013159 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cb148e-85e7-40d9-bbc5-21931b7b50cf None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:50.013432 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cb148e-85e7-40d9-bbc5-21931b7b50cf None admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:50.014216 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cb148e-85e7-40d9-bbc5-21931b7b50cf None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:50.014522 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cb148e-85e7-40d9-bbc5-21931b7b50cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:50.015941 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cb148e-85e7-40d9-bbc5-21931b7b50cf None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:50.040639 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:50.059609 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 254/508] 10.222.0.40 () {56 vars in 1137 bytes} [Fri Aug 5 15:12:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:50.059609 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cb148e-85e7-40d9-bbc5-21931b7b50cf None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:50.071654 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:50.095889 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a94008-29e8-4944-99a7-2464d0089134 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:50.131307 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9970c0-9377-4322-9e3e-15ab1127126f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:50.131942 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9970c0-9377-4322-9e3e-15ab1127126f None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:50.132261 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9970c0-9377-4322-9e3e-15ab1127126f None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:50.132539 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9970c0-9377-4322-9e3e-15ab1127126f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:50.142939 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9970c0-9377-4322-9e3e-15ab1127126f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:50.148396 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-9e9970c0-9377-4322-9e3e-15ab1127126f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:12:50.148396 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:50.148396 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:50.148396 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:50.148396 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:50.148396 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:50.148396 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:50.148396 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:50.148396 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:50.148396 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:50.148396 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:50.148396 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:50.148396 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:50.148396 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:50.148396 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:50.148396 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:50.149428 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:50.149428 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:50.149428 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:50.149428 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:50.149428 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:50.149428 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:50.149428 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:50.149428 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:50.149428 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:50.149428 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:50.149428 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:50.149428 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:50.149428 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:50.149428 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:50.149428 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:50.149428 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:50.150393 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:50.150393 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:50.150393 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:50.150393 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:50.150393 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:50.150393 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:50.150393 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:50.150393 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:50.150393 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:50.150393 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:50.150393 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:12:50.150393 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:50.150393 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 255/509] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 15:12:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:50.150393 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9970c0-9377-4322-9e3e-15ab1127126f None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:50.155258 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cb148e-85e7-40d9-bbc5-21931b7b50cf None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:50.192675 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdda82c2-072c-45a5-8f8b-b457820d1210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:50.193310 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda82c2-072c-45a5-8f8b-b457820d1210 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:50.193593 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda82c2-072c-45a5-8f8b-b457820d1210 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:50.193873 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda82c2-072c-45a5-8f8b-b457820d1210 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:50.200844 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdda82c2-072c-45a5-8f8b-b457820d1210 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:50.202299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdda82c2-072c-45a5-8f8b-b457820d1210 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:50.214380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-fdda82c2-072c-45a5-8f8b-b457820d1210 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 15:12:50.214380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:50.214380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:50.214380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:50.214380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:50.214380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:50.214380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:50.214380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:50.214380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:50.214380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:50.214380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:50.214380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:50.214380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 15:12:50.214380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 15:12:50.214380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 15:12:50.214380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 15:12:50.215446 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:50.215446 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:50.215446 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 15:12:50.215446 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:12:50.215446 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 15:12:50.215446 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:12:50.215446 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:50.215446 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:50.215446 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:50.215446 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:50.215446 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:50.215446 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:50.215446 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:50.215446 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:50.215446 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:50.215446 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:50.215446 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:50.215446 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:50.221063 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:50.221063 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:50.221063 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 15:12:50.221063 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 15:12:50.221063 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 15:12:50.221063 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 15:12:50.221063 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 15:12:50.221063 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 15:12:50.221063 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 15:12:50.221063 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:50.221063 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 255/510] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:12:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:50.221063 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdda82c2-072c-45a5-8f8b-b457820d1210 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:50.254014 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9970c0-9377-4322-9e3e-15ab1127126f None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:50.255541 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:50.287580 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7094a3a-8c93-42f1-ab61-9a36821b0372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:50.288258 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7094a3a-8c93-42f1-ab61-9a36821b0372 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:50.288544 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7094a3a-8c93-42f1-ab61-9a36821b0372 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:50.288825 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7094a3a-8c93-42f1-ab61-9a36821b0372 None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:50.289517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7094a3a-8c93-42f1-ab61-9a36821b0372 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:50.289836 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7094a3a-8c93-42f1-ab61-9a36821b0372 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:50.291207 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7094a3a-8c93-42f1-ab61-9a36821b0372 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:50.294817 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:50.303046 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 256/511] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:12:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:50.303833 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7094a3a-8c93-42f1-ab61-9a36821b0372 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:50.318468 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdda82c2-072c-45a5-8f8b-b457820d1210 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:50.352007 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff900c-865d-4b0c-9ef2-b08ea82e1d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:50.352683 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff900c-865d-4b0c-9ef2-b08ea82e1d83 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:50.352966 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff900c-865d-4b0c-9ef2-b08ea82e1d83 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:50.353241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff900c-865d-4b0c-9ef2-b08ea82e1d83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:50.363560 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efff900c-865d-4b0c-9ef2-b08ea82e1d83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:50.365040 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efff900c-865d-4b0c-9ef2-b08ea82e1d83 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:50.379638 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-efff900c-865d-4b0c-9ef2-b08ea82e1d83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:50.379638 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:50.379638 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:50.379638 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:50.379638 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:50.379638 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:50.379638 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:50.379638 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:50.379638 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:50.379638 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:50.379638 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:50.379638 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:50.379638 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:12:50.379638 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:12:50.379638 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:12:50.380754 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:12:50.380754 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:50.380754 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:50.380754 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:50.380754 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:50.380754 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:50.380754 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:50.380754 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:50.380754 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:50.380754 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:50.380754 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:50.380754 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:50.380754 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:50.380754 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:50.380754 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:50.380754 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:50.380754 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:50.382072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:12:50.382072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:12:50.382072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:50.382072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:50.382072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 256/512] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:12:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:50.382072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff900c-865d-4b0c-9ef2-b08ea82e1d83 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:50.405595 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7094a3a-8c93-42f1-ab61-9a36821b0372 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:50.439253 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f175b-7d61-4dc7-be3b-953272c559b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:50.439868 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f175b-7d61-4dc7-be3b-953272c559b4 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:50.440179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f175b-7d61-4dc7-be3b-953272c559b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:50.440459 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f175b-7d61-4dc7-be3b-953272c559b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:50.470179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:50.481122 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff900c-865d-4b0c-9ef2-b08ea82e1d83 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:50.507629 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:50.508686 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913f175b-7d61-4dc7-be3b-953272c559b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:50.510146 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913f175b-7d61-4dc7-be3b-953272c559b4 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:50.515675 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c35939e-5d77-43df-b486-a56c9ee89a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:50.516394 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c35939e-5d77-43df-b486-a56c9ee89a1b None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:50.516686 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c35939e-5d77-43df-b486-a56c9ee89a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:50.516967 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c35939e-5d77-43df-b486-a56c9ee89a1b None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:50.517734 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c35939e-5d77-43df-b486-a56c9ee89a1b None admin] RBAC: Adding query filter params (name=service) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:50.518040 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c35939e-5d77-43df-b486-a56c9ee89a1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:50.519984 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c35939e-5d77-43df-b486-a56c9ee89a1b None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:50.529450 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 257/513] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 15:12:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:50.530238 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c35939e-5d77-43df-b486-a56c9ee89a1b None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:50.627845 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 257/514] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:12:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 15:12:50.629198 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f175b-7d61-4dc7-be3b-953272c559b4 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:50.688552 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:50.692192 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c35939e-5d77-43df-b486-a56c9ee89a1b None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:50.725205 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c684b3-0306-4e80-b032-674d48dff7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:50.725813 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c684b3-0306-4e80-b032-674d48dff7a6 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:50.726097 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c684b3-0306-4e80-b032-674d48dff7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:50.726375 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c684b3-0306-4e80-b032-674d48dff7a6 None admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:50.726906 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f175b-7d61-4dc7-be3b-953272c559b4 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:50.727309 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c684b3-0306-4e80-b032-674d48dff7a6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:50.727456 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c684b3-0306-4e80-b032-674d48dff7a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:50.728907 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c684b3-0306-4e80-b032-674d48dff7a6 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:50.745823 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 258/515] 10.222.0.40 () {56 vars in 1137 bytes} [Fri Aug 5 15:12:50 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:50.746618 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c684b3-0306-4e80-b032-674d48dff7a6 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:50.765445 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415aee6-318e-475b-a173-0aa115a37340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:50.766074 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415aee6-318e-475b-a173-0aa115a37340 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:50.766353 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415aee6-318e-475b-a173-0aa115a37340 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:50.766624 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415aee6-318e-475b-a173-0aa115a37340 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:50.773146 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8415aee6-318e-475b-a173-0aa115a37340 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:50.774571 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8415aee6-318e-475b-a173-0aa115a37340 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:50.786248 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-8415aee6-318e-475b-a173-0aa115a37340 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:12:50.786248 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:50.786248 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:50.786248 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:50.786248 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:50.786248 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:50.786248 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:50.786248 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:50.786248 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:50.786248 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:50.786248 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:50.786248 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:50.786248 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 15:12:50.786248 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 15:12:50.786248 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 15:12:50.787280 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 15:12:50.787280 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:50.787280 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:50.787280 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:50.787280 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:50.787280 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:50.787280 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:50.787280 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:50.787280 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:50.787280 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:50.787280 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:50.787280 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:50.787280 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:50.787280 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:50.787280 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:50.787280 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:50.787280 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:50.788495 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 15:12:50.788495 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 15:12:50.788495 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 15:12:50.788495 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 15:12:50.788495 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 15:12:50.788495 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 15:12:50.788495 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:12:50.788495 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:50.789225 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 258/516] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:12:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:50.790069 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415aee6-318e-475b-a173-0aa115a37340 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:50.796015 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:50.846192 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c684b3-0306-4e80-b032-674d48dff7a6 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:50.880225 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f87a6-8c2f-4192-8631-98dd5d745094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:50.880765 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f87a6-8c2f-4192-8631-98dd5d745094 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:50.881050 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f87a6-8c2f-4192-8631-98dd5d745094 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:50.881330 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f87a6-8c2f-4192-8631-98dd5d745094 None admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:50.882003 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7f87a6-8c2f-4192-8631-98dd5d745094 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:50.882309 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7f87a6-8c2f-4192-8631-98dd5d745094 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:50.883669 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7f87a6-8c2f-4192-8631-98dd5d745094 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:50.894456 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 259/517] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:12:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:50.895223 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f87a6-8c2f-4192-8631-98dd5d745094 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:50.895747 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415aee6-318e-475b-a173-0aa115a37340 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:50.920502 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:50.932309 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78c2e87-60f6-44ca-9343-b23cae3ed650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:50.940604 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c2e87-60f6-44ca-9343-b23cae3ed650 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:50.940604 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c2e87-60f6-44ca-9343-b23cae3ed650 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:50.940604 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c2e87-60f6-44ca-9343-b23cae3ed650 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:50.943331 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78c2e87-60f6-44ca-9343-b23cae3ed650 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:50.945093 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78c2e87-60f6-44ca-9343-b23cae3ed650 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:50.959798 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-e78c2e87-60f6-44ca-9343-b23cae3ed650 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:50.959798 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:50.959798 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:50.959798 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:50.959798 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:50.959798 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:50.959798 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:50.959798 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:50.959798 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:50.959798 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:50.959798 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:50.959798 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:50.959798 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:12:50.959798 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:12:50.959798 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:12:50.960847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:12:50.960847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:50.960847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:50.960847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:50.960847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:50.960847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:50.960847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:50.960847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:50.960847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:50.960847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:50.960847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:50.960847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:50.960847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:50.960847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:50.960847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:50.960847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:50.960847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:50.966601 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:12:50.966601 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:12:50.966601 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:50.966601 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:50.966601 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 259/518] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:12:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:50.966601 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78c2e87-60f6-44ca-9343-b23cae3ed650 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:50.993527 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f87a6-8c2f-4192-8631-98dd5d745094 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:51.002476 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:51.026428 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5112e08b-7e89-4d90-8399-2a0b80cba14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:51.027239 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112e08b-7e89-4d90-8399-2a0b80cba14b None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:51.027685 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112e08b-7e89-4d90-8399-2a0b80cba14b None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:51.028289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112e08b-7e89-4d90-8399-2a0b80cba14b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:51.039125 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5112e08b-7e89-4d90-8399-2a0b80cba14b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:51.040637 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5112e08b-7e89-4d90-8399-2a0b80cba14b None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:51.055211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-5112e08b-7e89-4d90-8399-2a0b80cba14b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:51.055211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:51.055211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:51.055211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:51.055211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:51.055211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:51.055211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:51.055211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:51.055211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:51.055211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:51.055211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:51.055211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:51.055211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:12:51.055211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:12:51.055211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:12:51.056270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:12:51.056270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:51.056270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:51.056270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:51.056270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:51.056270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:51.056270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:51.056270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:51.056270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:51.056270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:51.056270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:51.056270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:51.056270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:51.056270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:51.056270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:51.056270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:51.056270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:51.061826 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:12:51.061826 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:12:51.061826 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:51.061826 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:51.061826 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 260/519] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:12:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:51.061826 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5112e08b-7e89-4d90-8399-2a0b80cba14b None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:51.062219 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78c2e87-60f6-44ca-9343-b23cae3ed650 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:51.093882 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f1a52-7ea5-479f-a4ac-1f27e9675ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:51.094494 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f1a52-7ea5-479f-a4ac-1f27e9675ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:51.094780 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f1a52-7ea5-479f-a4ac-1f27e9675ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:51.095075 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f1a52-7ea5-479f-a4ac-1f27e9675ed0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:51.137388 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:51.158288 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5112e08b-7e89-4d90-8399-2a0b80cba14b None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:51.163724 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f1a52-7ea5-479f-a4ac-1f27e9675ed0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:51.165284 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f1a52-7ea5-479f-a4ac-1f27e9675ed0 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:51.196513 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985155bb-01ec-436c-90ee-4a8c9c0c8b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:51.197212 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985155bb-01ec-436c-90ee-4a8c9c0c8b88 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:51.197504 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985155bb-01ec-436c-90ee-4a8c9c0c8b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:51.197787 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985155bb-01ec-436c-90ee-4a8c9c0c8b88 None admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:51.198534 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985155bb-01ec-436c-90ee-4a8c9c0c8b88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:51.198850 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985155bb-01ec-436c-90ee-4a8c9c0c8b88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:51.200312 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985155bb-01ec-436c-90ee-4a8c9c0c8b88 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:51.217881 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 261/520] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 15:12:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:51.218778 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985155bb-01ec-436c-90ee-4a8c9c0c8b88 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:51.240746 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:51.265074 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 260/521] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:12:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 15:12:51.266091 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f1a52-7ea5-479f-a4ac-1f27e9675ed0 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:51.318441 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985155bb-01ec-436c-90ee-4a8c9c0c8b88 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:51.340184 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:51.352802 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b15c95b-2e48-4e78-9011-590d51aa445b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:51.353414 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b15c95b-2e48-4e78-9011-590d51aa445b None admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:51.353704 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b15c95b-2e48-4e78-9011-590d51aa445b None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:51.353991 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b15c95b-2e48-4e78-9011-590d51aa445b None admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:51.354664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b15c95b-2e48-4e78-9011-590d51aa445b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:51.356131 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b15c95b-2e48-4e78-9011-590d51aa445b None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:51.365352 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f1a52-7ea5-479f-a4ac-1f27e9675ed0 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:51.399822 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec73a79-756e-42cc-b1ea-0851bda6bff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:51.400391 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec73a79-756e-42cc-b1ea-0851bda6bff2 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:51.400670 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec73a79-756e-42cc-b1ea-0851bda6bff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:51.400966 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec73a79-756e-42cc-b1ea-0851bda6bff2 None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:51.401645 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec73a79-756e-42cc-b1ea-0851bda6bff2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:51.401946 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec73a79-756e-42cc-b1ea-0851bda6bff2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:51.403424 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec73a79-756e-42cc-b1ea-0851bda6bff2 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:51.414818 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 261/522] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:12:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:51.415595 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec73a79-756e-42cc-b1ea-0851bda6bff2 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:51.416056 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 262/523] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 5 15:12:51 2022] POST /identity/v3/users => generated 247 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:12:51.417085 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b15c95b-2e48-4e78-9011-590d51aa445b None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:51.507812 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:51.536743 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b15c95b-2e48-4e78-9011-590d51aa445b None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:51.545694 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec73a79-756e-42cc-b1ea-0851bda6bff2 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:51.577939 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c224ed5-7665-45fe-b61b-152f1fa0c71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:51.578563 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c224ed5-7665-45fe-b61b-152f1fa0c71d None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:51.578851 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c224ed5-7665-45fe-b61b-152f1fa0c71d None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:51.579128 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c224ed5-7665-45fe-b61b-152f1fa0c71d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:51.579915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c224ed5-7665-45fe-b61b-152f1fa0c71d None admin] RBAC: Adding query filter params (role.id=266a023f66e043f99efe2941fe58d481, scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83, user.id=bc0497dab1f44bb7b8196d39bf01d770) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:51.580370 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c224ed5-7665-45fe-b61b-152f1fa0c71d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:51.582836 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c224ed5-7665-45fe-b61b-152f1fa0c71d None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:51.587030 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee0cf0-ebfd-4a07-b6a8-32f111554ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:51.587650 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee0cf0-ebfd-4a07-b6a8-32f111554ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:51.587929 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee0cf0-ebfd-4a07-b6a8-32f111554ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:51.588241 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee0cf0-ebfd-4a07-b6a8-32f111554ea0 None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:51.588941 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdee0cf0-ebfd-4a07-b6a8-32f111554ea0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:51.589277 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdee0cf0-ebfd-4a07-b6a8-32f111554ea0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:51.592176 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 263/524] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:12:51 2022] GET /identity/v3/role_assignments?user.id=bc0497dab1f44bb7b8196d39bf01d770&scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83&role.id=266a023f66e043f99efe2941fe58d481 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:51.592317 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdee0cf0-ebfd-4a07-b6a8-32f111554ea0 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:51.592395 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c224ed5-7665-45fe-b61b-152f1fa0c71d None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:51.593030 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:51.609720 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 262/525] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:12:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:51.610523 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee0cf0-ebfd-4a07-b6a8-32f111554ea0 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:51.727252 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c224ed5-7665-45fe-b61b-152f1fa0c71d None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:51.740702 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee0cf0-ebfd-4a07-b6a8-32f111554ea0 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:51.767853 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f6231-a2ef-4681-93ba-dd6a1328ba64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:51.769611 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f6231-a2ef-4681-93ba-dd6a1328ba64 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:51.770091 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f6231-a2ef-4681-93ba-dd6a1328ba64 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:51.770539 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f6231-a2ef-4681-93ba-dd6a1328ba64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:51.780670 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:51.783144 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef6e33c-61f9-481c-98b4-3ccaad4fca93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:51.783288 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4f6231-a2ef-4681-93ba-dd6a1328ba64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:51.784112 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6e33c-61f9-481c-98b4-3ccaad4fca93 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:51.784576 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6e33c-61f9-481c-98b4-3ccaad4fca93 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:51.785019 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6e33c-61f9-481c-98b4-3ccaad4fca93 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:51.785131 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4f6231-a2ef-4681-93ba-dd6a1328ba64 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:51.793384 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef6e33c-61f9-481c-98b4-3ccaad4fca93 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:51.795127 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef6e33c-61f9-481c-98b4-3ccaad4fca93 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:51.801036 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-ca4f6231-a2ef-4681-93ba-dd6a1328ba64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:51.801036 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:51.801036 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:51.801036 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:51.801036 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:51.801036 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:51.801036 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:51.801036 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:51.801036 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:51.801036 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:51.801036 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:51.801036 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:51.801036 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:12:51.801036 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:12:51.801036 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:12:51.802120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:12:51.802120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:51.802120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:51.802120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:51.802120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:51.802120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:51.802120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:51.802120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:51.802120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:51.802120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:51.802120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:51.802120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:51.802120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:51.802120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:51.802120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:51.802120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:51.802120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:51.803225 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:12:51.803225 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:12:51.803225 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:51.803225 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:51.804744 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 264/526] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:12:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:51.809912 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f6231-a2ef-4681-93ba-dd6a1328ba64 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:51.811473 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-7ef6e33c-61f9-481c-98b4-3ccaad4fca93 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 15:12:51.811473 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:51.811473 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:51.811473 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:51.811473 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:51.811473 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:51.811473 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:51.811473 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:51.811473 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:51.811473 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:51.811473 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:51.811473 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:51.811473 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 15:12:51.811473 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 15:12:51.811473 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 15:12:51.811473 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 15:12:51.812537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:51.812537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:51.812537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 15:12:51.812537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:12:51.812537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 15:12:51.812537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:12:51.812537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:51.812537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:51.812537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:51.812537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:51.812537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:51.812537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:51.812537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:51.812537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:51.812537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:51.812537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:51.812537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:51.812537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:51.813459 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:51.813459 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:51.813459 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 15:12:51.813459 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 15:12:51.813459 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 15:12:51.813459 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 15:12:51.813459 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 15:12:51.813459 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 15:12:51.813459 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 15:12:51.813459 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:51.815460 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 263/527] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 15:12:51 2022] GET /identity/v3/users/neutron => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:51.817163 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef6e33c-61f9-481c-98b4-3ccaad4fca93 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:51.879952 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:51.927939 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef6e33c-61f9-481c-98b4-3ccaad4fca93 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:51.950933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f6231-a2ef-4681-93ba-dd6a1328ba64 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:51.969421 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d47a6-14fe-4b40-bfaf-5c711740ac18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:51.970314 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d47a6-14fe-4b40-bfaf-5c711740ac18 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:51.970544 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d47a6-14fe-4b40-bfaf-5c711740ac18 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:51.970918 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d47a6-14fe-4b40-bfaf-5c711740ac18 None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:51.971723 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693d47a6-14fe-4b40-bfaf-5c711740ac18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:51.972172 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693d47a6-14fe-4b40-bfaf-5c711740ac18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:51.973723 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693d47a6-14fe-4b40-bfaf-5c711740ac18 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:51.990344 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 264/528] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:12:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:51.991308 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d47a6-14fe-4b40-bfaf-5c711740ac18 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:51.994569 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8229080d-d46f-48d5-8d45-02dc77d6761d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:51.996398 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229080d-d46f-48d5-8d45-02dc77d6761d None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:51.996830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229080d-d46f-48d5-8d45-02dc77d6761d None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:51.997276 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229080d-d46f-48d5-8d45-02dc77d6761d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:52.007191 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8229080d-d46f-48d5-8d45-02dc77d6761d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:52.009551 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8229080d-d46f-48d5-8d45-02dc77d6761d None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:52.026260 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-8229080d-d46f-48d5-8d45-02dc77d6761d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:12:52.026260 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:52.026260 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:52.026260 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:52.026260 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:52.026260 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:52.026260 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:52.026260 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:52.026260 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:52.026260 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:52.026260 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:52.026260 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:52.026260 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 15:12:52.026260 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 15:12:52.026260 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 15:12:52.027295 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 15:12:52.027295 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:52.027295 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:52.027295 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:52.027295 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:52.027295 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:52.027295 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:52.027295 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:52.027295 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:52.027295 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:52.027295 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:52.027295 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:52.027295 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:52.027295 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:52.027295 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:52.027295 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:52.027295 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:52.028354 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 15:12:52.028354 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 15:12:52.028354 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 15:12:52.028354 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 15:12:52.028354 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 15:12:52.028354 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 15:12:52.028354 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:12:52.028354 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:52.031395 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 265/529] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:12:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:52.032488 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8229080d-d46f-48d5-8d45-02dc77d6761d None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:52.100652 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d47a6-14fe-4b40-bfaf-5c711740ac18 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:52.103962 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:52.120795 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:52.149212 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0141b639-4f48-4635-a8aa-2a99b8776da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:52.149852 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141b639-4f48-4635-a8aa-2a99b8776da5 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:52.150139 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141b639-4f48-4635-a8aa-2a99b8776da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:52.150420 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141b639-4f48-4635-a8aa-2a99b8776da5 None admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:52.151507 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0141b639-4f48-4635-a8aa-2a99b8776da5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:52.151823 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0141b639-4f48-4635-a8aa-2a99b8776da5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:52.154067 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0141b639-4f48-4635-a8aa-2a99b8776da5 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:52.170154 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8229080d-d46f-48d5-8d45-02dc77d6761d None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:52.187425 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 265/530] 10.222.0.40 () {56 vars in 1139 bytes} [Fri Aug 5 15:12:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:52.188412 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0141b639-4f48-4635-a8aa-2a99b8776da5 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:52.208968 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8920e6c6-478c-4c7f-b149-ca90f4e8f6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:52.209936 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920e6c6-478c-4c7f-b149-ca90f4e8f6b5 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:52.210412 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920e6c6-478c-4c7f-b149-ca90f4e8f6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:52.210898 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920e6c6-478c-4c7f-b149-ca90f4e8f6b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:52.218374 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8920e6c6-478c-4c7f-b149-ca90f4e8f6b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:52.220189 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8920e6c6-478c-4c7f-b149-ca90f4e8f6b5 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:52.233353 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-8920e6c6-478c-4c7f-b149-ca90f4e8f6b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:12:52.233353 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:52.233353 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:52.233353 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:52.233353 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:52.233353 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:52.233353 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:52.233353 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:52.233353 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:52.233353 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:52.233353 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:52.233353 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:52.233353 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 15:12:52.233353 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 15:12:52.233353 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 15:12:52.234223 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 15:12:52.234223 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:52.234223 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:52.234223 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:52.234223 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:52.234223 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:52.234223 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:52.234223 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:52.234223 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:52.234223 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:52.234223 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:52.234223 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:52.234223 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:52.234223 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:52.234223 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:52.234223 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:52.234223 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:52.235058 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 15:12:52.235058 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 15:12:52.235058 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 15:12:52.235058 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 15:12:52.235058 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 15:12:52.235058 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 15:12:52.235058 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:12:52.235058 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:52.238177 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 266/531] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:12:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:52.239232 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8920e6c6-478c-4c7f-b149-ca90f4e8f6b5 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:52.318048 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0141b639-4f48-4635-a8aa-2a99b8776da5 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:52.354428 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb50b7b-bc87-4c5a-b122-c31e3af01601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:52.355374 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb50b7b-bc87-4c5a-b122-c31e3af01601 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:52.355873 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb50b7b-bc87-4c5a-b122-c31e3af01601 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:52.356540 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb50b7b-bc87-4c5a-b122-c31e3af01601 None admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:52.357475 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb50b7b-bc87-4c5a-b122-c31e3af01601 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:52.358013 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb50b7b-bc87-4c5a-b122-c31e3af01601 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:52.359560 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8920e6c6-478c-4c7f-b149-ca90f4e8f6b5 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:52.359873 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb50b7b-bc87-4c5a-b122-c31e3af01601 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:52.374945 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:52.380393 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 266/532] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 15:12:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:52.381538 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb50b7b-bc87-4c5a-b122-c31e3af01601 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:52.402591 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fbedc4-7b04-4794-b2dd-8fec9595fd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:52.404141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fbedc4-7b04-4794-b2dd-8fec9595fd13 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:52.404793 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fbedc4-7b04-4794-b2dd-8fec9595fd13 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:52.405088 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fbedc4-7b04-4794-b2dd-8fec9595fd13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:52.418254 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:52.489886 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fbedc4-7b04-4794-b2dd-8fec9595fd13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:52.491698 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fbedc4-7b04-4794-b2dd-8fec9595fd13 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:52.493009 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb50b7b-bc87-4c5a-b122-c31e3af01601 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:52.529204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6134f5-b649-48a0-8171-341627566a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:52.529868 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6134f5-b649-48a0-8171-341627566a13 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:52.530175 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6134f5-b649-48a0-8171-341627566a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:52.530454 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6134f5-b649-48a0-8171-341627566a13 None admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:52.531202 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6134f5-b649-48a0-8171-341627566a13 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:52.531516 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6134f5-b649-48a0-8171-341627566a13 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:52.532973 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6134f5-b649-48a0-8171-341627566a13 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:52.545408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 267/533] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 15:12:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:52.546461 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6134f5-b649-48a0-8171-341627566a13 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:52.601326 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 267/534] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:12:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 15:12:52.602624 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fbedc4-7b04-4794-b2dd-8fec9595fd13 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:52.618770 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:52.648565 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:52.652723 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6134f5-b649-48a0-8171-341627566a13 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:52.686787 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47280212-550c-415f-ab98-55b2f72b23ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:52.687406 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47280212-550c-415f-ab98-55b2f72b23ba None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:52.687688 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47280212-550c-415f-ab98-55b2f72b23ba None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:52.687961 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47280212-550c-415f-ab98-55b2f72b23ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:52.688816 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47280212-550c-415f-ab98-55b2f72b23ba None admin] RBAC: Adding query filter params (role.id=266a023f66e043f99efe2941fe58d481, scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83, user.id=c386fa0483cf42efaddbc2395f6d1fce) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:52.689119 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47280212-550c-415f-ab98-55b2f72b23ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:52.690559 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47280212-550c-415f-ab98-55b2f72b23ba None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:52.697274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 268/535] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:12:52 2022] GET /identity/v3/role_assignments?user.id=c386fa0483cf42efaddbc2395f6d1fce&scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83&role.id=266a023f66e043f99efe2941fe58d481 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:52.698077 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47280212-550c-415f-ab98-55b2f72b23ba None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:52.718153 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fbedc4-7b04-4794-b2dd-8fec9595fd13 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:52.758263 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31555a7a-0eab-4134-b5c7-369228d23328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:52.759861 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31555a7a-0eab-4134-b5c7-369228d23328 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:52.760412 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31555a7a-0eab-4134-b5c7-369228d23328 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:52.760886 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31555a7a-0eab-4134-b5c7-369228d23328 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:52.762193 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31555a7a-0eab-4134-b5c7-369228d23328 None admin] RBAC: Adding query filter params (role.id=266a023f66e043f99efe2941fe58d481, scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83, user.id=e974f8d9a06d489884a3b00b95dd88e4) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:52.762726 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31555a7a-0eab-4134-b5c7-369228d23328 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:52.764565 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31555a7a-0eab-4134-b5c7-369228d23328 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:52.772481 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 268/536] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:12:52 2022] GET /identity/v3/role_assignments?user.id=e974f8d9a06d489884a3b00b95dd88e4&scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83&role.id=266a023f66e043f99efe2941fe58d481 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:52.831381 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47280212-550c-415f-ab98-55b2f72b23ba None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:52.877925 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b92055-291c-4daf-9dd4-b82ce1f63b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:52.879102 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b92055-291c-4daf-9dd4-b82ce1f63b2d None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:52.879385 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b92055-291c-4daf-9dd4-b82ce1f63b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:52.879658 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b92055-291c-4daf-9dd4-b82ce1f63b2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:52.901788 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b92055-291c-4daf-9dd4-b82ce1f63b2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:52.902267 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b92055-291c-4daf-9dd4-b82ce1f63b2d None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:52.923520 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-d1b92055-291c-4daf-9dd4-b82ce1f63b2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:52.923520 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:52.923520 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:52.923520 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:52.923520 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:52.923520 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:52.923520 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:52.923520 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:52.923520 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:52.923520 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:52.923520 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:52.923520 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:52.923520 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:12:52.923520 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:12:52.923520 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:12:52.924672 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:12:52.924672 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:52.924672 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:52.924672 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:52.924672 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:52.924672 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:52.924672 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:52.924672 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:52.924672 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:52.924672 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:52.924672 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:52.924672 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:52.924672 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:52.924672 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:52.924672 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:52.924672 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:52.924672 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:52.926071 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:12:52.926071 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:12:52.926071 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:52.926071 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:52.928385 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 269/537] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:12:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:52.934112 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31555a7a-0eab-4134-b5c7-369228d23328 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:53.018153 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:53.042629 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31555a7a-0eab-4134-b5c7-369228d23328 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:53.078810 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c072b050-bf44-469e-a267-0d8424d660a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:53.079767 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c072b050-bf44-469e-a267-0d8424d660a7 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:53.080282 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c072b050-bf44-469e-a267-0d8424d660a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:53.080753 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c072b050-bf44-469e-a267-0d8424d660a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:53.081706 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c072b050-bf44-469e-a267-0d8424d660a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:53.082210 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c072b050-bf44-469e-a267-0d8424d660a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:53.083774 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c072b050-bf44-469e-a267-0d8424d660a7 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:53.096262 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 269/538] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:12:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:53.105137 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b92055-291c-4daf-9dd4-b82ce1f63b2d None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:53.119166 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:53.222663 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b92055-291c-4daf-9dd4-b82ce1f63b2d None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:53.257864 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c7d658-63e1-4366-8d87-3593abdf3d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:53.258906 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c7d658-63e1-4366-8d87-3593abdf3d0e None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:53.259480 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c7d658-63e1-4366-8d87-3593abdf3d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:53.259952 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c7d658-63e1-4366-8d87-3593abdf3d0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:53.267285 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c7d658-63e1-4366-8d87-3593abdf3d0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:53.269128 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c7d658-63e1-4366-8d87-3593abdf3d0e None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:53.281035 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-99c7d658-63e1-4366-8d87-3593abdf3d0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:12:53.281035 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:53.281035 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:53.281035 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:53.281035 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:53.281035 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:53.281035 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:53.281035 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:53.281035 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:53.281035 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:53.281035 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:53.281035 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:53.281035 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 15:12:53.281035 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 15:12:53.281035 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 15:12:53.281931 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 15:12:53.281931 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:53.281931 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:53.281931 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:53.281931 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:53.281931 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:53.281931 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:53.281931 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:53.281931 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:53.281931 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:53.281931 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:53.281931 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:53.281931 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:53.281931 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:53.281931 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:53.281931 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:53.281931 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:53.282755 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 15:12:53.282755 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 15:12:53.282755 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 15:12:53.282755 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 15:12:53.282755 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 15:12:53.282755 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 15:12:53.282755 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:12:53.282755 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:53.285787 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 270/539] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:12:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:53.293206 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c072b050-bf44-469e-a267-0d8424d660a7 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:53.443493 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c072b050-bf44-469e-a267-0d8424d660a7 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:53.484090 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb8badf-30cf-4bb1-ad48-7162b7212edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:53.486028 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8badf-30cf-4bb1-ad48-7162b7212edd None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:53.486316 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8badf-30cf-4bb1-ad48-7162b7212edd None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:53.486595 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8badf-30cf-4bb1-ad48-7162b7212edd None admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:53.487354 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb8badf-30cf-4bb1-ad48-7162b7212edd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:53.488258 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb8badf-30cf-4bb1-ad48-7162b7212edd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:53.490478 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb8badf-30cf-4bb1-ad48-7162b7212edd None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:53.495786 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:53.513202 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 270/540] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 15:12:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:53.519139 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c7d658-63e1-4366-8d87-3593abdf3d0e None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:53.617450 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:53.651632 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c7d658-63e1-4366-8d87-3593abdf3d0e None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:53.688251 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a227ea-147b-43ab-8062-a0f18bf69328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:53.688871 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a227ea-147b-43ab-8062-a0f18bf69328 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:53.689154 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a227ea-147b-43ab-8062-a0f18bf69328 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:53.689426 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a227ea-147b-43ab-8062-a0f18bf69328 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:53.690229 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a227ea-147b-43ab-8062-a0f18bf69328 None admin] RBAC: Adding query filter params (role.id=266a023f66e043f99efe2941fe58d481, scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83, user.id=d13a1638b2ca497fb02c71615559d1ea) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:53.690559 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a227ea-147b-43ab-8062-a0f18bf69328 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:53.691998 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a227ea-147b-43ab-8062-a0f18bf69328 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:53.701819 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 271/541] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:12:53 2022] GET /identity/v3/role_assignments?user.id=d13a1638b2ca497fb02c71615559d1ea&scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83&role.id=266a023f66e043f99efe2941fe58d481 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:53.807911 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108364d1-eb36-4f7b-9ec3-b27d86bf64c8 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:53.808704 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108364d1-eb36-4f7b-9ec3-b27d86bf64c8 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:53.808704 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108364d1-eb36-4f7b-9ec3-b27d86bf64c8 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:53.810248 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 271/542] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:53.817342 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c2622a-ed56-4415-a9eb-8d6ede4efe1d None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:53.817880 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c2622a-ed56-4415-a9eb-8d6ede4efe1d None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:53.818405 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c2622a-ed56-4415-a9eb-8d6ede4efe1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:53.834043 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150a216-0011-4d1c-9172-8938c43f0be7 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:53.834350 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150a216-0011-4d1c-9172-8938c43f0be7 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:53.834618 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150a216-0011-4d1c-9172-8938c43f0be7 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:53.836228 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 272/543] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:53.841509 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c83e64-5795-45ad-b183-e836666ff52e None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:53.842168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c83e64-5795-45ad-b183-e836666ff52e None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:53.842650 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c83e64-5795-45ad-b183-e836666ff52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:53.844226 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-65c2622a-ed56-4415-a9eb-8d6ede4efe1d None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:53.845479 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-65c2622a-ed56-4415-a9eb-8d6ede4efe1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:53.881349 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-d4c83e64-5795-45ad-b183-e836666ff52e None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:53.884332 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-d4c83e64-5795-45ad-b183-e836666ff52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:53.957154 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 272/544] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:53.976881 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c2622a-ed56-4415-a9eb-8d6ede4efe1d None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:54.004515 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 273/545] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:54.037896 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c83e64-5795-45ad-b183-e836666ff52e None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:54.062332 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:54.127654 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c2622a-ed56-4415-a9eb-8d6ede4efe1d None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:54.181191 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebd8e49-7d4e-43a9-9d1a-fa331050123b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:54.182169 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd8e49-7d4e-43a9-9d1a-fa331050123b None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:54.182844 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd8e49-7d4e-43a9-9d1a-fa331050123b None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:54.184009 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd8e49-7d4e-43a9-9d1a-fa331050123b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:54.205080 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebd8e49-7d4e-43a9-9d1a-fa331050123b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:54.215846 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-3ebd8e49-7d4e-43a9-9d1a-fa331050123b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:12:54.215846 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:54.215846 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:54.215846 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:54.215846 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:54.215846 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:54.215846 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:54.215846 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:54.215846 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:54.215846 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:54.215846 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:54.215846 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:54.215846 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:54.215846 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:54.215846 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:54.215846 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:54.216758 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:54.216758 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:54.216758 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:54.216758 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:54.216758 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:54.216758 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:54.216758 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:54.216758 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:54.216758 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:54.216758 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:54.216758 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:54.216758 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:54.216758 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:54.216758 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:54.216758 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:54.216758 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:54.217552 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:54.217552 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:54.217552 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:54.217552 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:54.217552 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:54.217552 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:54.217552 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:54.217552 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:54.217552 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:54.217552 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:54.217552 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:12:54.217552 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:54.221661 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 273/546] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 15:12:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:12:54.228156 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c83e64-5795-45ad-b183-e836666ff52e None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:54.232207 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebd8e49-7d4e-43a9-9d1a-fa331050123b None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:54.285773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:54.307823 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80d134-36ca-4f47-887f-f37471ef067c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:54.308487 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80d134-36ca-4f47-887f-f37471ef067c None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:54.308773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80d134-36ca-4f47-887f-f37471ef067c None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:54.309054 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80d134-36ca-4f47-887f-f37471ef067c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:54.332444 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d80d134-36ca-4f47-887f-f37471ef067c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:54.345651 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-0d80d134-36ca-4f47-887f-f37471ef067c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:12:54.345651 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:54.345651 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:54.345651 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:54.345651 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:54.345651 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:54.345651 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:54.345651 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:54.345651 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:54.345651 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:54.345651 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:54.345651 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:54.345651 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:54.345651 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:54.345651 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:54.345651 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:54.346613 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:54.346613 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:54.346613 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:54.346613 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:54.346613 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:54.346613 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:54.346613 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:54.346613 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:54.346613 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:54.346613 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:54.346613 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:54.346613 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:54.346613 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:54.346613 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:54.346613 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:54.346613 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:54.347459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:54.347459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:54.347459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:54.347459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:54.347459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:54.347459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:54.347459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:54.347459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:54.347459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:54.347459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:54.347459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:12:54.347459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:54.347459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 274/547] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 15:12:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:54.353038 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80d134-36ca-4f47-887f-f37471ef067c None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:54.363131 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebd8e49-7d4e-43a9-9d1a-fa331050123b None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:54.392571 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:54.401166 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b05096-6438-4930-a1e6-8572d9a25ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:54.402128 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b05096-6438-4930-a1e6-8572d9a25ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:54.402636 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b05096-6438-4930-a1e6-8572d9a25ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:54.403096 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b05096-6438-4930-a1e6-8572d9a25ae6 None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:54.404285 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b05096-6438-4930-a1e6-8572d9a25ae6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:54.404777 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b05096-6438-4930-a1e6-8572d9a25ae6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:54.406516 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b05096-6438-4930-a1e6-8572d9a25ae6 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:54.415873 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 274/548] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 15:12:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:54.422055 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b05096-6438-4930-a1e6-8572d9a25ae6 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:54.535783 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b05096-6438-4930-a1e6-8572d9a25ae6 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:54.548340 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80d134-36ca-4f47-887f-f37471ef067c None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:54.573281 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb80de-04f2-4775-a0ce-2ca207e4fee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:54.574385 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb80de-04f2-4775-a0ce-2ca207e4fee0 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:54.574904 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb80de-04f2-4775-a0ce-2ca207e4fee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:54.575399 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb80de-04f2-4775-a0ce-2ca207e4fee0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:54.587931 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeb80de-04f2-4775-a0ce-2ca207e4fee0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:54.589812 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeb80de-04f2-4775-a0ce-2ca207e4fee0 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:54.605942 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-5aeb80de-04f2-4775-a0ce-2ca207e4fee0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:54.605942 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:54.605942 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:54.605942 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:54.605942 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:54.605942 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:54.605942 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:54.605942 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:54.605942 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:54.605942 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:54.605942 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:54.605942 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:54.605942 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:12:54.605942 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:12:54.605942 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:12:54.607021 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:12:54.607021 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:54.607021 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:54.607021 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:54.607021 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:54.607021 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:54.607021 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:54.607021 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:54.607021 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:54.607021 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:54.607021 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:54.607021 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:54.607021 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:54.607021 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:54.607021 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:54.607021 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:54.607021 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:54.607898 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:12:54.607898 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:12:54.607898 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:54.607898 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:54.610216 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 275/549] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:12:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:54.616621 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb80de-04f2-4775-a0ce-2ca207e4fee0 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:54.668502 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:54.696764 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99ee75-7eba-4015-a139-7c33ba667137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:54.697400 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99ee75-7eba-4015-a139-7c33ba667137 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:54.697687 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99ee75-7eba-4015-a139-7c33ba667137 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:54.697968 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99ee75-7eba-4015-a139-7c33ba667137 None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:54.698690 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b99ee75-7eba-4015-a139-7c33ba667137 None admin] RBAC: Adding query filter params (name=service) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:54.698999 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b99ee75-7eba-4015-a139-7c33ba667137 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:54.700423 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b99ee75-7eba-4015-a139-7c33ba667137 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:54.726270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 275/550] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 15:12:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:54.727842 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb80de-04f2-4775-a0ce-2ca207e4fee0 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:54.732992 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99ee75-7eba-4015-a139-7c33ba667137 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:54.765017 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbbc5be-dafd-40e3-b562-6fd548848556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:54.765979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbbc5be-dafd-40e3-b562-6fd548848556 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:54.766469 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbbc5be-dafd-40e3-b562-6fd548848556 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:54.766972 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbbc5be-dafd-40e3-b562-6fd548848556 None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:54.767866 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbbc5be-dafd-40e3-b562-6fd548848556 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:54.768423 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbbc5be-dafd-40e3-b562-6fd548848556 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:54.770271 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbbc5be-dafd-40e3-b562-6fd548848556 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:54.783161 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 276/551] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:12:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:12:54.789763 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbbc5be-dafd-40e3-b562-6fd548848556 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:54.821084 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:54.900933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99ee75-7eba-4015-a139-7c33ba667137 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:54.920447 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:54.929232 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbbc5be-dafd-40e3-b562-6fd548848556 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:54.967374 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b067b-ac07-4420-94f9-213df52a7d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:54.968022 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b067b-ac07-4420-94f9-213df52a7d96 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:54.968391 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b067b-ac07-4420-94f9-213df52a7d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:54.968642 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b067b-ac07-4420-94f9-213df52a7d96 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:54.978038 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481b067b-ac07-4420-94f9-213df52a7d96 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:54.979662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481b067b-ac07-4420-94f9-213df52a7d96 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:54.992530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-481b067b-ac07-4420-94f9-213df52a7d96 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 15:12:54.992530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:54.992530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:54.992530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:54.992530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:54.992530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:54.992530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:54.992530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:54.992530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:54.992530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:54.992530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:54.992530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:54.992530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 15:12:54.992530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 15:12:54.992530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 15:12:54.992530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 15:12:54.993501 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:54.993501 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:54.993501 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 15:12:54.993501 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:12:54.993501 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 15:12:54.993501 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:12:54.993501 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:54.993501 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:54.993501 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:54.993501 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:54.993501 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:54.993501 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:54.993501 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:54.993501 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:54.993501 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:54.993501 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:54.993501 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:54.993501 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:54.994416 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:54.994416 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:54.994416 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 15:12:54.994416 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 15:12:54.994416 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 15:12:54.994416 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 15:12:54.994416 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 15:12:54.994416 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 15:12:54.994416 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 15:12:54.994416 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:54.994416 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 277/552] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Aug 5 15:12:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:54.999805 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b067b-ac07-4420-94f9-213df52a7d96 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:55.000251 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56124f6-f685-49b6-b1bb-c35eeddbf36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:55.000916 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56124f6-f685-49b6-b1bb-c35eeddbf36f None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:55.001202 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56124f6-f685-49b6-b1bb-c35eeddbf36f None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:55.001482 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56124f6-f685-49b6-b1bb-c35eeddbf36f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:55.035394 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56124f6-f685-49b6-b1bb-c35eeddbf36f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:55.036893 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56124f6-f685-49b6-b1bb-c35eeddbf36f None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:55.070004 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-b56124f6-f685-49b6-b1bb-c35eeddbf36f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:55.070004 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:55.070004 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:55.070004 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:55.070004 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:55.070004 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:55.070004 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:55.070004 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:55.070004 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:55.070004 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:55.070004 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:55.070004 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:55.070004 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:12:55.070004 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:12:55.070004 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:12:55.070988 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:12:55.070988 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:55.070988 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:55.070988 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:55.070988 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:55.070988 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:55.070988 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:55.070988 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:55.070988 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:55.070988 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:55.070988 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:55.070988 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:55.070988 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:55.070988 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:55.070988 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:55.070988 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:55.070988 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:55.071846 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:12:55.071846 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:12:55.071846 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:55.071846 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:55.076002 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 276/553] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:12:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:55.085087 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56124f6-f685-49b6-b1bb-c35eeddbf36f None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:55.111012 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b067b-ac07-4420-94f9-213df52a7d96 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:55.147050 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3943417-2fa9-418e-bbbd-a65f5fc6454c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:55.147727 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3943417-2fa9-418e-bbbd-a65f5fc6454c None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:55.148020 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3943417-2fa9-418e-bbbd-a65f5fc6454c None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:55.148332 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3943417-2fa9-418e-bbbd-a65f5fc6454c None admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:55.149201 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3943417-2fa9-418e-bbbd-a65f5fc6454c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:55.149631 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3943417-2fa9-418e-bbbd-a65f5fc6454c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:55.151179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3943417-2fa9-418e-bbbd-a65f5fc6454c None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:55.181213 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 278/554] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Aug 5 15:12:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:12:55.186663 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3943417-2fa9-418e-bbbd-a65f5fc6454c None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:55.220444 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:55.290755 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:55.293729 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56124f6-f685-49b6-b1bb-c35eeddbf36f None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:55.308425 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3943417-2fa9-418e-bbbd-a65f5fc6454c None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:55.342525 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f57492-5512-4388-beb2-9434a7e0a7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:55.343070 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f57492-5512-4388-beb2-9434a7e0a7fe None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:55.343362 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f57492-5512-4388-beb2-9434a7e0a7fe None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:55.343649 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f57492-5512-4388-beb2-9434a7e0a7fe None admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:55.344385 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f57492-5512-4388-beb2-9434a7e0a7fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:55.344697 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f57492-5512-4388-beb2-9434a7e0a7fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:55.346075 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f57492-5512-4388-beb2-9434a7e0a7fe None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:55.346399 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d6f589-a1d6-4b11-9912-f7e544fee9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:55.347997 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6f589-a1d6-4b11-9912-f7e544fee9aa None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:55.348552 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6f589-a1d6-4b11-9912-f7e544fee9aa None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:55.349053 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6f589-a1d6-4b11-9912-f7e544fee9aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:55.360574 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d6f589-a1d6-4b11-9912-f7e544fee9aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:55.362321 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d6f589-a1d6-4b11-9912-f7e544fee9aa None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:55.365406 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 277/555] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:12:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:55.366840 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abae1e7-c174-43a2-bf57-81837e4fe167 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:55.367133 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abae1e7-c174-43a2-bf57-81837e4fe167 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:55.367418 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abae1e7-c174-43a2-bf57-81837e4fe167 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:55.368934 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 278/556] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:55.370214 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abae1e7-c174-43a2-bf57-81837e4fe167 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:55.386098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-60d6f589-a1d6-4b11-9912-f7e544fee9aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:55.386098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:55.386098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:55.386098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:55.386098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:55.386098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:55.386098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:55.386098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:55.386098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:55.386098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:55.386098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:55.386098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:55.386098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:12:55.386098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:12:55.386098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:12:55.386972 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:12:55.386972 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:55.386972 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:55.386972 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:55.386972 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:55.386972 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:55.386972 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:55.386972 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:55.386972 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:55.386972 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:55.386972 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:55.386972 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:55.386972 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:55.386972 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:55.386972 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:55.386972 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:55.386972 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:55.388356 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:12:55.388356 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:12:55.388356 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:55.388356 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:55.390614 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 279/557] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:12:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:12:55.392177 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b5c1c-cb5e-48da-ba23-c1011bb3ec92 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:55.392708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b5c1c-cb5e-48da-ba23-c1011bb3ec92 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:55.393231 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b5c1c-cb5e-48da-ba23-c1011bb3ec92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:55.413639 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-916b5c1c-cb5e-48da-ba23-c1011bb3ec92 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:55.415147 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-916b5c1c-cb5e-48da-ba23-c1011bb3ec92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:55.481007 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:55.484741 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abae1e7-c174-43a2-bf57-81837e4fe167 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:55.490208 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 280/558] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:55.491293 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b5c1c-cb5e-48da-ba23-c1011bb3ec92 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:55.538931 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f5ca57-f299-4386-b8a5-766357467075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:55.539931 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5ca57-f299-4386-b8a5-766357467075 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:55.540309 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5ca57-f299-4386-b8a5-766357467075 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:55.540657 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5ca57-f299-4386-b8a5-766357467075 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:55.550518 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f5ca57-f299-4386-b8a5-766357467075 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:55.552934 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f5ca57-f299-4386-b8a5-766357467075 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:55.569470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-f5f5ca57-f299-4386-b8a5-766357467075 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 15:12:55.569470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:55.569470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:55.569470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:55.569470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:55.569470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:55.569470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:55.569470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:55.569470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:55.569470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:55.569470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:55.569470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:55.569470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 15:12:55.569470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 15:12:55.569470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 15:12:55.569470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 15:12:55.570588 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:55.570588 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:55.570588 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 15:12:55.570588 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:12:55.570588 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 15:12:55.570588 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:12:55.570588 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:55.570588 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:55.570588 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:55.570588 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:55.570588 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:55.570588 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:55.570588 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:55.570588 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:55.570588 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:55.570588 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:55.570588 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:55.570588 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:55.571782 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:55.571782 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:55.571782 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 15:12:55.571782 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 15:12:55.571782 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 15:12:55.571782 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 15:12:55.571782 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 15:12:55.571782 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 15:12:55.571782 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 15:12:55.571782 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:55.572653 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 279/559] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 15:12:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:55.574022 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e3367-42db-4f2b-99f0-9e78abc2952a None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:55.574308 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e3367-42db-4f2b-99f0-9e78abc2952a None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:55.574586 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e3367-42db-4f2b-99f0-9e78abc2952a None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:55.575914 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 280/560] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:55.576641 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082e3367-42db-4f2b-99f0-9e78abc2952a None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:55.599511 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:55.606225 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b5c1c-cb5e-48da-ba23-c1011bb3ec92 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:55.643260 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0325ca0b-f73c-405a-8920-26a9be99941b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:55.644597 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0325ca0b-f73c-405a-8920-26a9be99941b None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:55.645008 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0325ca0b-f73c-405a-8920-26a9be99941b None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:55.645388 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0325ca0b-f73c-405a-8920-26a9be99941b None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:55.646187 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0325ca0b-f73c-405a-8920-26a9be99941b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:55.646595 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0325ca0b-f73c-405a-8920-26a9be99941b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:55.648044 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0325ca0b-f73c-405a-8920-26a9be99941b None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:55.660807 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 281/561] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:12:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:55.661772 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0325ca0b-f73c-405a-8920-26a9be99941b None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:55.686617 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082e3367-42db-4f2b-99f0-9e78abc2952a None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:55.720544 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:55.725871 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd232f-56e5-4694-abd1-4842247acf53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:55.726509 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd232f-56e5-4694-abd1-4842247acf53 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:55.726798 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd232f-56e5-4694-abd1-4842247acf53 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:55.727078 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd232f-56e5-4694-abd1-4842247acf53 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:55.738813 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecd232f-56e5-4694-abd1-4842247acf53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:55.744531 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-3ecd232f-56e5-4694-abd1-4842247acf53 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:12:55.744531 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:55.744531 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:55.744531 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:55.744531 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:55.744531 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:55.744531 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:55.744531 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:55.744531 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:55.744531 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:55.744531 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:55.744531 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:55.744531 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:55.744531 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:55.744531 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:55.744531 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:55.745558 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:55.745558 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:55.745558 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:55.745558 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:55.745558 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:55.745558 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:55.745558 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:55.745558 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:55.745558 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:55.745558 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:55.745558 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:55.745558 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:55.745558 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:55.745558 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:55.745558 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:55.745558 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:55.746404 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:55.746404 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:55.746404 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:55.746404 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:55.746404 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:55.746404 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:55.746404 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:55.746404 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:55.746404 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:55.746404 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:55.746404 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:12:55.746404 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:55.746404 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 281/562] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 15:12:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:55.747092 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2686f2-7ba8-426f-bf3b-84f23253c046 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:55.747168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2686f2-7ba8-426f-bf3b-84f23253c046 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:55.747733 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2686f2-7ba8-426f-bf3b-84f23253c046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:55.768361 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-ac2686f2-7ba8-426f-bf3b-84f23253c046 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:55.770437 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-ac2686f2-7ba8-426f-bf3b-84f23253c046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:55.772150 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0325ca0b-f73c-405a-8920-26a9be99941b None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:55.808018 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedbb295-2fd5-4aca-86a4-2424f937b224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:55.808752 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedbb295-2fd5-4aca-86a4-2424f937b224 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:55.809107 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedbb295-2fd5-4aca-86a4-2424f937b224 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:55.809451 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedbb295-2fd5-4aca-86a4-2424f937b224 None admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:55.810324 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedbb295-2fd5-4aca-86a4-2424f937b224 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:55.810717 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedbb295-2fd5-4aca-86a4-2424f937b224 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:55.812251 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedbb295-2fd5-4aca-86a4-2424f937b224 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:55.835029 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 282/563] 10.222.0.40 () {56 vars in 1135 bytes} [Fri Aug 5 15:12:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:55.835029 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedbb295-2fd5-4aca-86a4-2424f937b224 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:55.848105 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 282/564] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:55.848105 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2686f2-7ba8-426f-bf3b-84f23253c046 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:55.868786 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:55.950117 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedbb295-2fd5-4aca-86a4-2424f937b224 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:55.966834 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:55.969824 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2686f2-7ba8-426f-bf3b-84f23253c046 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:55.989414 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff599e11-2f6c-463a-a759-73e718f95ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:55.990064 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff599e11-2f6c-463a-a759-73e718f95ae0 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:55.990349 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff599e11-2f6c-463a-a759-73e718f95ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:55.990632 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff599e11-2f6c-463a-a759-73e718f95ae0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:55.997620 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff599e11-2f6c-463a-a759-73e718f95ae0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:55.999318 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff599e11-2f6c-463a-a759-73e718f95ae0 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:56.009059 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc5fc1e-1382-40a9-afbd-14ff12d1ffa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:56.009936 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5fc1e-1382-40a9-afbd-14ff12d1ffa0 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:56.010426 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5fc1e-1382-40a9-afbd-14ff12d1ffa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:56.010925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5fc1e-1382-40a9-afbd-14ff12d1ffa0 None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:56.011133 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-ff599e11-2f6c-463a-a759-73e718f95ae0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:12:56.011133 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:56.011133 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:56.011133 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:56.011133 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:56.011133 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:56.011133 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:56.011133 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:56.011133 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:56.011133 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:56.011133 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:56.011133 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:56.011133 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 15:12:56.011133 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 15:12:56.011133 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 15:12:56.011943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 15:12:56.011943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:56.011943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:56.011943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:56.011943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:56.011943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:56.011943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:56.011943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:56.011943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:56.011943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:56.011943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:56.011943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:56.011943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:56.011943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:56.011943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:56.011943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:56.011943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:56.014086 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 15:12:56.014086 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 15:12:56.014086 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 15:12:56.014086 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 15:12:56.014086 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 15:12:56.014086 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 15:12:56.014086 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:12:56.014086 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:56.014086 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 283/565] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:12:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:56.014562 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc5fc1e-1382-40a9-afbd-14ff12d1ffa0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:56.014562 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc5fc1e-1382-40a9-afbd-14ff12d1ffa0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:56.014693 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc5fc1e-1382-40a9-afbd-14ff12d1ffa0 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:56.017072 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff599e11-2f6c-463a-a759-73e718f95ae0 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:56.024652 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 283/566] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 15:12:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:56.028120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc5fc1e-1382-40a9-afbd-14ff12d1ffa0 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:56.106451 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:56.120816 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff599e11-2f6c-463a-a759-73e718f95ae0 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:56.136868 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc5fc1e-1382-40a9-afbd-14ff12d1ffa0 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:56.155268 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ceee73-72d4-4b01-b3b5-452599c60bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:56.155875 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ceee73-72d4-4b01-b3b5-452599c60bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:56.156189 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ceee73-72d4-4b01-b3b5-452599c60bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:56.156477 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ceee73-72d4-4b01-b3b5-452599c60bf6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:56.176043 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c58978-3a60-41ba-ab64-19e2d904c2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:56.176770 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c58978-3a60-41ba-ab64-19e2d904c2a4 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:56.177058 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c58978-3a60-41ba-ab64-19e2d904c2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:56.177335 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c58978-3a60-41ba-ab64-19e2d904c2a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:56.187507 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c58978-3a60-41ba-ab64-19e2d904c2a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:56.192901 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-f0c58978-3a60-41ba-ab64-19e2d904c2a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:12:56.192901 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:56.192901 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:56.192901 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:56.192901 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:56.192901 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:56.192901 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:56.192901 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:56.192901 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:56.192901 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:56.192901 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:56.192901 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:56.192901 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:56.192901 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:56.192901 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:56.192901 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:56.193986 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:56.193986 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:56.193986 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:56.193986 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:56.193986 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:56.193986 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:56.193986 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:56.193986 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:56.193986 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:56.193986 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:56.193986 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:56.193986 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:56.193986 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:56.193986 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:56.193986 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:56.193986 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:56.195003 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:56.195003 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:56.195003 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:56.195003 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:56.195003 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:56.195003 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:56.195003 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:56.195003 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:56.195003 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:56.195003 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:56.195003 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:12:56.195003 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:56.195003 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 284/567] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 15:12:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:56.195003 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c58978-3a60-41ba-ab64-19e2d904c2a4 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:56.204470 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:56.239347 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ceee73-72d4-4b01-b3b5-452599c60bf6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:56.241217 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ceee73-72d4-4b01-b3b5-452599c60bf6 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:56.296123 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c58978-3a60-41ba-ab64-19e2d904c2a4 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:56.324864 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:56.329863 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9ebf2-0160-4d42-b2fe-897839b41ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:56.330479 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9ebf2-0160-4d42-b2fe-897839b41ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:56.330759 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9ebf2-0160-4d42-b2fe-897839b41ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:56.331080 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9ebf2-0160-4d42-b2fe-897839b41ad0 None admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:56.331821 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b9ebf2-0160-4d42-b2fe-897839b41ad0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:56.332164 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b9ebf2-0160-4d42-b2fe-897839b41ad0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:56.333711 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b9ebf2-0160-4d42-b2fe-897839b41ad0 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:56.338379 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 284/568] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:12:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 15:12:56.339573 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ceee73-72d4-4b01-b3b5-452599c60bf6 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:56.350298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 285/569] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 15:12:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:56.350298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e064a7-29a4-4bb3-8fec-b1da0e0632cd None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:56.350298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e064a7-29a4-4bb3-8fec-b1da0e0632cd None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:56.350298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e064a7-29a4-4bb3-8fec-b1da0e0632cd None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:56.350298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 286/570] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:56.350298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e064a7-29a4-4bb3-8fec-b1da0e0632cd None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:56.417024 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:56.441315 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ceee73-72d4-4b01-b3b5-452599c60bf6 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:56.449769 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e064a7-29a4-4bb3-8fec-b1da0e0632cd None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:56.474889 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b14fc45-249c-4980-b088-69e0be407561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:56.475521 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b14fc45-249c-4980-b088-69e0be407561 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:56.475798 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b14fc45-249c-4980-b088-69e0be407561 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:56.476093 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b14fc45-249c-4980-b088-69e0be407561 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:56.482425 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69f6e54-ec94-4465-a78f-b7e141f1d1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:56.483020 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f6e54-ec94-4465-a78f-b7e141f1d1b3 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:56.483301 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f6e54-ec94-4465-a78f-b7e141f1d1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:56.483576 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f6e54-ec94-4465-a78f-b7e141f1d1b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:56.484304 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69f6e54-ec94-4465-a78f-b7e141f1d1b3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:56.484609 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69f6e54-ec94-4465-a78f-b7e141f1d1b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:56.485744 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b14fc45-249c-4980-b088-69e0be407561 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:56.486062 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69f6e54-ec94-4465-a78f-b7e141f1d1b3 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:56.487165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b14fc45-249c-4980-b088-69e0be407561 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:56.494339 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 287/571] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 15:12:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:56.495295 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69f6e54-ec94-4465-a78f-b7e141f1d1b3 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:56.505656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-3b14fc45-249c-4980-b088-69e0be407561 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:56.505656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:56.505656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:56.505656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:56.505656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:56.505656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:56.505656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:56.505656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:56.505656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:56.505656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:56.505656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:56.505656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:56.505656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:12:56.505656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:12:56.505656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:12:56.510043 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:12:56.510043 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:56.510043 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:56.510043 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:56.510043 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:56.510043 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:56.510043 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:56.510043 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:56.510043 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:56.510043 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:56.510043 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:56.510043 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:56.510043 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:56.510043 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:56.510043 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:56.510043 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:56.510043 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:56.511692 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:12:56.511692 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:12:56.511692 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:56.511692 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:56.511692 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 285/572] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:12:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:56.511692 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b14fc45-249c-4980-b088-69e0be407561 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:56.563036 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:56.607357 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69f6e54-ec94-4465-a78f-b7e141f1d1b3 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:56.615405 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b14fc45-249c-4980-b088-69e0be407561 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:56.639949 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdde713-47ec-47cf-aba0-31c6df38a4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:56.640138 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:56.640439 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdde713-47ec-47cf-aba0-31c6df38a4a7 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:56.640730 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdde713-47ec-47cf-aba0-31c6df38a4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:56.641008 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdde713-47ec-47cf-aba0-31c6df38a4a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:56.649289 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c0bd8d-547b-4544-b523-965b4267b878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:56.649954 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c0bd8d-547b-4544-b523-965b4267b878 None admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:56.652163 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c0bd8d-547b-4544-b523-965b4267b878 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:56.652163 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c0bd8d-547b-4544-b523-965b4267b878 None admin] PATH_INFO: `/v3/projects/06f27ab8886a4c0d81c4d4e3fa100a83/users/bc0497dab1f44bb7b8196d39bf01d770/roles/266a023f66e043f99efe2941fe58d481` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:56.652356 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdde713-47ec-47cf-aba0-31c6df38a4a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:56.653606 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdde713-47ec-47cf-aba0-31c6df38a4a7 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:56.665325 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c0bd8d-547b-4544-b523-965b4267b878 None admin] RBAC: Authorizing `identity:create_grant(project_id=06f27ab8886a4c0d81c4d4e3fa100a83, user_id=bc0497dab1f44bb7b8196d39bf01d770, role_id=266a023f66e043f99efe2941fe58d481)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:56.667224 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c0bd8d-547b-4544-b523-965b4267b878 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:56.668328 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-fbdde713-47ec-47cf-aba0-31c6df38a4a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:56.668328 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:56.668328 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:56.668328 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:56.668328 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:56.668328 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:56.668328 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:56.668328 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:56.668328 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:56.668328 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:56.668328 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:56.668328 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:56.668328 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:12:56.668328 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:12:56.668328 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:12:56.669354 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:12:56.669354 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:56.669354 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:56.669354 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:56.669354 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:56.669354 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:56.669354 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:56.669354 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:56.669354 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:56.669354 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:56.669354 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:56.669354 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:56.669354 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:56.669354 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:56.669354 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:56.669354 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:56.669354 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:56.670275 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:12:56.670275 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:12:56.670275 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:56.670275 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:56.670275 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 288/573] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:12:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:56.670595 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a554bdf-6084-4881-8988-385824471dcd None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:56.670874 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a554bdf-6084-4881-8988-385824471dcd None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:56.671146 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a554bdf-6084-4881-8988-385824471dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:56.688500 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-7a554bdf-6084-4881-8988-385824471dcd None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:56.689699 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-7a554bdf-6084-4881-8988-385824471dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:56.752226 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 286/574] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 15:12:56 2022] PUT /identity/v3/projects/06f27ab8886a4c0d81c4d4e3fa100a83/users/bc0497dab1f44bb7b8196d39bf01d770/roles/266a023f66e043f99efe2941fe58d481 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:12:56.753053 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c0bd8d-547b-4544-b523-965b4267b878 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:56.755665 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 289/575] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:56.756397 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a554bdf-6084-4881-8988-385824471dcd None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:56.807185 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:56.882896 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a554bdf-6084-4881-8988-385824471dcd None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:56.883585 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c0bd8d-547b-4544-b523-965b4267b878 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:56.926276 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcbe998-8605-4c7d-ab0f-0701f0bc6a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:56.926826 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbe998-8605-4c7d-ab0f-0701f0bc6a42 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:56.927174 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbe998-8605-4c7d-ab0f-0701f0bc6a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:56.927401 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbe998-8605-4c7d-ab0f-0701f0bc6a42 None admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:56.928214 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcbe998-8605-4c7d-ab0f-0701f0bc6a42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:56.928442 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcbe998-8605-4c7d-ab0f-0701f0bc6a42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:56.929786 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0265432-2395-4cbd-ba46-9a4b1cac5ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:56.929900 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcbe998-8605-4c7d-ab0f-0701f0bc6a42 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:56.930437 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0265432-2395-4cbd-ba46-9a4b1cac5ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:56.930777 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0265432-2395-4cbd-ba46-9a4b1cac5ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:56.931060 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0265432-2395-4cbd-ba46-9a4b1cac5ef6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:56.944489 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 290/576] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:12:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:56.944489 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcbe998-8605-4c7d-ab0f-0701f0bc6a42 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:56.947781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0265432-2395-4cbd-ba46-9a4b1cac5ef6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:56.949359 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0265432-2395-4cbd-ba46-9a4b1cac5ef6 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:56.966031 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-b0265432-2395-4cbd-ba46-9a4b1cac5ef6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:56.966031 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:56.966031 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:56.966031 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:56.966031 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:56.966031 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:56.966031 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:56.966031 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:56.966031 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:56.966031 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:56.966031 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:56.966031 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:56.966031 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:12:56.966031 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:12:56.966031 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:12:56.967408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:12:56.967408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:56.967408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:56.967408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:56.967408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:56.967408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:56.967408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:56.967408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:56.967408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:56.967408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:56.967408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:56.967408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:56.967408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:56.967408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:56.967408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:56.967408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:56.967408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:56.968365 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:12:56.968365 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:12:56.968365 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:56.968365 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:56.968365 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 287/577] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:12:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:56.968365 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0265432-2395-4cbd-ba46-9a4b1cac5ef6 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:56.996700 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:57.051506 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:57.057319 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcbe998-8605-4c7d-ab0f-0701f0bc6a42 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:57.072240 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0265432-2395-4cbd-ba46-9a4b1cac5ef6 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:57.092127 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f137433-4c3f-43d2-a05b-deb4a57eee2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:57.092749 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f137433-4c3f-43d2-a05b-deb4a57eee2c None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:57.093037 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f137433-4c3f-43d2-a05b-deb4a57eee2c None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:57.093353 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f137433-4c3f-43d2-a05b-deb4a57eee2c None admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:57.094060 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f137433-4c3f-43d2-a05b-deb4a57eee2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:57.094375 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f137433-4c3f-43d2-a05b-deb4a57eee2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:57.095748 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f137433-4c3f-43d2-a05b-deb4a57eee2c None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:57.108520 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 291/578] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:12:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:57.109012 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a344105-a9d8-441a-9984-92419a716ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:57.109438 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f137433-4c3f-43d2-a05b-deb4a57eee2c None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:57.110051 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a344105-a9d8-441a-9984-92419a716ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:57.110343 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a344105-a9d8-441a-9984-92419a716ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:57.110625 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a344105-a9d8-441a-9984-92419a716ea0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:57.123977 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a344105-a9d8-441a-9984-92419a716ea0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:57.129775 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-6a344105-a9d8-441a-9984-92419a716ea0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:12:57.129775 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:57.129775 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:57.129775 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:57.129775 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:57.129775 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:57.129775 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:57.129775 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:57.129775 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:57.129775 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:57.129775 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:57.129775 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:57.129775 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:57.129775 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:57.129775 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:57.129775 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:57.130696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:57.130696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:57.130696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:57.130696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:57.130696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:57.130696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:57.130696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:57.130696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:57.130696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:57.130696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:57.130696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:57.130696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:57.130696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:57.130696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:57.130696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:57.130696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:57.131554 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:57.131554 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:57.131554 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:57.131554 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:57.131554 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:57.131554 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:57.131554 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:57.131554 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:57.131554 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:57.131554 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:57.131554 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:12:57.131554 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:57.131554 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 288/579] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 15:12:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:57.132909 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a344105-a9d8-441a-9984-92419a716ea0 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:57.227233 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f137433-4c3f-43d2-a05b-deb4a57eee2c None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:57.233066 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:57.245864 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a344105-a9d8-441a-9984-92419a716ea0 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:57.267257 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe52edd-61e2-402a-a826-5bada885e1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:57.267899 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe52edd-61e2-402a-a826-5bada885e1f2 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:57.268314 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe52edd-61e2-402a-a826-5bada885e1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:57.268533 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe52edd-61e2-402a-a826-5bada885e1f2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:57.276255 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe52edd-61e2-402a-a826-5bada885e1f2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:57.278051 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe52edd-61e2-402a-a826-5bada885e1f2 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:57.279707 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31480244-cab6-47b1-9cda-c1ef17843d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:57.280362 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31480244-cab6-47b1-9cda-c1ef17843d46 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:57.280650 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31480244-cab6-47b1-9cda-c1ef17843d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:57.280931 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31480244-cab6-47b1-9cda-c1ef17843d46 None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:57.281628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31480244-cab6-47b1-9cda-c1ef17843d46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:57.282014 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31480244-cab6-47b1-9cda-c1ef17843d46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:57.283413 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31480244-cab6-47b1-9cda-c1ef17843d46 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:57.292681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-fbe52edd-61e2-402a-a826-5bada885e1f2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 05 15:12:57.292681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:57.292681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:57.292681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:57.292681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:57.292681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:57.292681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:57.292681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:57.292681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:57.292681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:57.292681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:57.292681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:57.292681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 15:12:57.292681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 15:12:57.292681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 15:12:57.292681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 15:12:57.293630 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:57.293630 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:57.293630 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 15:12:57.293630 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:12:57.293630 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 15:12:57.293630 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:12:57.293630 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:57.293630 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:57.293630 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:57.293630 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:57.293630 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:57.293630 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:57.293630 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:57.293630 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:57.293630 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:57.293630 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:57.293630 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:57.293630 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:57.294652 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:57.294652 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:57.294652 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 15:12:57.294652 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 15:12:57.294652 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 15:12:57.294652 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 15:12:57.294652 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 15:12:57.294652 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 15:12:57.294652 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 05 15:12:57.294652 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:57.302367 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 292/580] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:12:57 2022] GET /identity/v3/users/glance => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:57.302367 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe52edd-61e2-402a-a826-5bada885e1f2 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:57.302576 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 289/581] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:12:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:57.302576 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31480244-cab6-47b1-9cda-c1ef17843d46 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:57.321336 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:57.406133 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31480244-cab6-47b1-9cda-c1ef17843d46 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:57.406566 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe52edd-61e2-402a-a826-5bada885e1f2 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:57.441018 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c70449-bc24-47bf-ad89-c787bc8abea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:57.441708 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c70449-bc24-47bf-ad89-c787bc8abea5 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:57.441984 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c70449-bc24-47bf-ad89-c787bc8abea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:57.442267 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c70449-bc24-47bf-ad89-c787bc8abea5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:57.443722 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e2f2b-9213-445b-baa6-60a63c08287d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:57.444638 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e2f2b-9213-445b-baa6-60a63c08287d None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:57.445165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e2f2b-9213-445b-baa6-60a63c08287d None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:57.445587 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e2f2b-9213-445b-baa6-60a63c08287d None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:57.446471 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064e2f2b-9213-445b-baa6-60a63c08287d None admin] RBAC: Adding query filter params (name=service) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:57.446925 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064e2f2b-9213-445b-baa6-60a63c08287d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:57.448488 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064e2f2b-9213-445b-baa6-60a63c08287d None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:57.449286 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c70449-bc24-47bf-ad89-c787bc8abea5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:57.450734 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c70449-bc24-47bf-ad89-c787bc8abea5 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:57.458009 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 290/582] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 15:12:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:57.459013 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e2f2b-9213-445b-baa6-60a63c08287d None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:57.463028 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-01c70449-bc24-47bf-ad89-c787bc8abea5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:12:57.463028 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:57.463028 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:57.463028 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:57.463028 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:57.463028 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:57.463028 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:57.463028 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:57.463028 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:57.463028 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:57.463028 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:57.463028 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:57.463028 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 15:12:57.463028 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 15:12:57.463028 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 15:12:57.463907 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 15:12:57.463907 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:57.463907 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:57.463907 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:57.463907 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:57.463907 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:57.463907 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:57.463907 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:57.463907 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:57.463907 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:57.463907 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:57.463907 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:57.463907 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:57.463907 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:57.463907 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:57.463907 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:57.463907 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:57.464851 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 15:12:57.464851 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 15:12:57.464851 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 15:12:57.464851 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 15:12:57.464851 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 15:12:57.464851 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 15:12:57.464851 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:12:57.464851 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:57.468378 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 293/583] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:12:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:57.469258 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c70449-bc24-47bf-ad89-c787bc8abea5 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:57.476655 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:57.553501 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:57.575650 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c70449-bc24-47bf-ad89-c787bc8abea5 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:57.622032 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e2f2b-9213-445b-baa6-60a63c08287d None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:57.623698 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d933b9-2822-4798-9d40-89d3f1a5a70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:57.624355 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d933b9-2822-4798-9d40-89d3f1a5a70b None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:57.624638 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d933b9-2822-4798-9d40-89d3f1a5a70b None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:57.624921 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d933b9-2822-4798-9d40-89d3f1a5a70b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:57.633504 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d933b9-2822-4798-9d40-89d3f1a5a70b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:57.634962 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d933b9-2822-4798-9d40-89d3f1a5a70b None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:57.646807 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-d6d933b9-2822-4798-9d40-89d3f1a5a70b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 15:12:57.646807 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:57.646807 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:57.646807 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:57.646807 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:57.646807 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:57.646807 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:57.646807 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:57.646807 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:57.646807 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:57.646807 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:57.646807 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:57.646807 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 15:12:57.646807 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 15:12:57.646807 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 15:12:57.646807 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 15:12:57.647759 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:57.647759 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:57.647759 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 15:12:57.647759 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:12:57.647759 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 15:12:57.647759 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:12:57.647759 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:57.647759 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:57.647759 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:57.647759 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:57.647759 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:57.647759 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:57.647759 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:57.647759 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:57.647759 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:57.647759 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:57.647759 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:57.647759 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:57.649056 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:57.649056 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:57.649056 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 15:12:57.649056 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 15:12:57.649056 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 15:12:57.649056 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 15:12:57.649056 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 15:12:57.649056 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 15:12:57.649056 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 15:12:57.649056 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:57.649056 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 294/584] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:12:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:57.649056 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d933b9-2822-4798-9d40-89d3f1a5a70b None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:57.664580 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a64f9-1122-478c-af81-4c9c3836d62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:57.665195 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a64f9-1122-478c-af81-4c9c3836d62e None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:57.665481 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a64f9-1122-478c-af81-4c9c3836d62e None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:57.665760 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a64f9-1122-478c-af81-4c9c3836d62e None admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:57.666582 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7a64f9-1122-478c-af81-4c9c3836d62e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:57.666892 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7a64f9-1122-478c-af81-4c9c3836d62e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:57.668349 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7a64f9-1122-478c-af81-4c9c3836d62e None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:57.685317 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 291/585] 10.222.0.40 () {56 vars in 1137 bytes} [Fri Aug 5 15:12:57 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:57.686096 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a64f9-1122-478c-af81-4c9c3836d62e None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:57.757687 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:57.760737 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d933b9-2822-4798-9d40-89d3f1a5a70b None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:57.793091 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a64f9-1122-478c-af81-4c9c3836d62e None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:57.799248 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24746d-9c21-425f-96be-83bd1acdf92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:57.799930 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24746d-9c21-425f-96be-83bd1acdf92b None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:57.800252 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24746d-9c21-425f-96be-83bd1acdf92b None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:57.800530 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24746d-9c21-425f-96be-83bd1acdf92b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:57.803658 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:57.812860 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b24746d-9c21-425f-96be-83bd1acdf92b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:57.816876 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b24746d-9c21-425f-96be-83bd1acdf92b None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:57.829445 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200f2ec-16c6-4bf9-b39f-9ab84ef49056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:57.829768 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-8b24746d-9c21-425f-96be-83bd1acdf92b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:57.829768 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:57.829768 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:57.829768 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:57.829768 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:57.829768 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:57.829768 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:57.829768 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:57.829768 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:57.829768 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:57.829768 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:57.829768 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:57.829768 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:12:57.829768 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:12:57.829768 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:12:57.830614 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:12:57.830614 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:57.830614 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:57.830614 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:57.830614 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:57.830614 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:57.830614 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:57.830614 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:57.830614 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:57.830614 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:57.830614 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:57.830614 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:57.830614 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:57.830614 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:57.830614 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:57.830614 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:57.830614 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:57.832500 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:12:57.832500 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:12:57.832500 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:57.832500 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:57.837499 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200f2ec-16c6-4bf9-b39f-9ab84ef49056 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:57.837499 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200f2ec-16c6-4bf9-b39f-9ab84ef49056 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:57.837499 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200f2ec-16c6-4bf9-b39f-9ab84ef49056 None admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:57.837499 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c200f2ec-16c6-4bf9-b39f-9ab84ef49056 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:57.837499 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c200f2ec-16c6-4bf9-b39f-9ab84ef49056 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:57.837841 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 295/586] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:12:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:57.837841 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24746d-9c21-425f-96be-83bd1acdf92b None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:57.838040 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c200f2ec-16c6-4bf9-b39f-9ab84ef49056 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:57.849755 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 292/587] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 15:12:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:57.850708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200f2ec-16c6-4bf9-b39f-9ab84ef49056 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:57.948860 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24746d-9c21-425f-96be-83bd1acdf92b None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:57.966470 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200f2ec-16c6-4bf9-b39f-9ab84ef49056 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:57.990010 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f3bd2c-40ba-4de4-ba83-71ff6953a6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:57.990710 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3bd2c-40ba-4de4-ba83-71ff6953a6a4 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:57.990998 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3bd2c-40ba-4de4-ba83-71ff6953a6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:57.991276 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3bd2c-40ba-4de4-ba83-71ff6953a6a4 None admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:57.992176 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f3bd2c-40ba-4de4-ba83-71ff6953a6a4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:57.992494 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f3bd2c-40ba-4de4-ba83-71ff6953a6a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:57.993923 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f3bd2c-40ba-4de4-ba83-71ff6953a6a4 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:57.999631 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:58.008844 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176ff4f7-f1c3-49ca-9461-7a99dfb44dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:58.016583 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ff4f7-f1c3-49ca-9461-7a99dfb44dbd None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:58.016583 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ff4f7-f1c3-49ca-9461-7a99dfb44dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:58.016583 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ff4f7-f1c3-49ca-9461-7a99dfb44dbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:58.016838 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 296/588] 10.222.0.40 () {56 vars in 1137 bytes} [Fri Aug 5 15:12:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:58.016838 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f3bd2c-40ba-4de4-ba83-71ff6953a6a4 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:58.026740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176ff4f7-f1c3-49ca-9461-7a99dfb44dbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:58.028268 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176ff4f7-f1c3-49ca-9461-7a99dfb44dbd None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:58.045121 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-176ff4f7-f1c3-49ca-9461-7a99dfb44dbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:58.045121 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:58.045121 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:58.045121 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:58.045121 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:58.045121 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:58.045121 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:58.045121 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:58.045121 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:58.045121 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:58.045121 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:58.045121 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:58.045121 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:12:58.045121 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:12:58.045121 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:12:58.046184 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:12:58.046184 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:58.046184 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:58.046184 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:58.046184 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:58.046184 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:58.046184 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:58.046184 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:58.046184 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:58.046184 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:58.046184 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:58.046184 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:58.046184 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:58.046184 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:58.046184 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:58.046184 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:58.046184 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:58.047134 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:12:58.047134 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:12:58.047134 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:58.047134 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:58.047134 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 293/589] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:12:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:58.047134 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176ff4f7-f1c3-49ca-9461-7a99dfb44dbd None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:58.067265 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:58.129538 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f3bd2c-40ba-4de4-ba83-71ff6953a6a4 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:58.152717 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176ff4f7-f1c3-49ca-9461-7a99dfb44dbd None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:58.166001 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c382e0f-b345-4adf-ac18-960bd1ed2991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:58.169199 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c382e0f-b345-4adf-ac18-960bd1ed2991 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:58.169199 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c382e0f-b345-4adf-ac18-960bd1ed2991 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:58.169199 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c382e0f-b345-4adf-ac18-960bd1ed2991 None admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:58.169199 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c382e0f-b345-4adf-ac18-960bd1ed2991 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:58.169199 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c382e0f-b345-4adf-ac18-960bd1ed2991 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:58.169574 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c382e0f-b345-4adf-ac18-960bd1ed2991 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:58.183290 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 297/590] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:12:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:58.184353 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c382e0f-b345-4adf-ac18-960bd1ed2991 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:58.199539 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc20bf7-b89d-430f-89c9-2847f2e9a05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:58.200203 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc20bf7-b89d-430f-89c9-2847f2e9a05e None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:58.200490 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc20bf7-b89d-430f-89c9-2847f2e9a05e None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:58.200773 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc20bf7-b89d-430f-89c9-2847f2e9a05e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:58.201566 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc20bf7-b89d-430f-89c9-2847f2e9a05e None admin] RBAC: Adding query filter params (role.id=266a023f66e043f99efe2941fe58d481, scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83, user.id=85154ce7446245738edcc8702dbf6861) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:58.201878 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc20bf7-b89d-430f-89c9-2847f2e9a05e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:58.203305 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc20bf7-b89d-430f-89c9-2847f2e9a05e None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:58.208789 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 294/591] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:12:58 2022] GET /identity/v3/role_assignments?user.id=85154ce7446245738edcc8702dbf6861&scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83&role.id=266a023f66e043f99efe2941fe58d481 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:58.209558 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc20bf7-b89d-430f-89c9-2847f2e9a05e None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:58.270522 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:58.333524 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c382e0f-b345-4adf-ac18-960bd1ed2991 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:58.343145 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:58.355554 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc20bf7-b89d-430f-89c9-2847f2e9a05e None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:58.376400 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b417078c-3538-481f-a3d6-e66948d54a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:58.376620 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b417078c-3538-481f-a3d6-e66948d54a91 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:58.378426 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b417078c-3538-481f-a3d6-e66948d54a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:58.378426 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b417078c-3538-481f-a3d6-e66948d54a91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:58.393580 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b417078c-3538-481f-a3d6-e66948d54a91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:58.395140 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b417078c-3538-481f-a3d6-e66948d54a91 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:58.397144 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084c2f9e-6839-4271-9375-6c0472316d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:58.397977 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c2f9e-6839-4271-9375-6c0472316d4d None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:58.398655 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c2f9e-6839-4271-9375-6c0472316d4d None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:58.399185 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c2f9e-6839-4271-9375-6c0472316d4d None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:58.400030 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084c2f9e-6839-4271-9375-6c0472316d4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:58.400591 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084c2f9e-6839-4271-9375-6c0472316d4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:58.402191 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084c2f9e-6839-4271-9375-6c0472316d4d None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:58.413024 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-b417078c-3538-481f-a3d6-e66948d54a91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:58.413024 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:58.413024 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:58.413024 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:58.413024 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:58.413024 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:58.413024 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:58.413024 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:58.413024 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:58.413024 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:58.413024 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:58.413024 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:58.413024 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:12:58.413024 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:12:58.413024 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:12:58.414134 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:12:58.414134 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:58.414134 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:58.414134 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:58.414134 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:58.414134 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:58.414134 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:58.414134 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:58.414134 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:58.414134 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:58.414134 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:58.414134 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:58.414134 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:58.414134 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:58.414134 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:58.414134 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:58.414134 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:58.415255 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:12:58.415255 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:12:58.415255 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:58.415255 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:58.416919 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 298/592] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:12:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:58.418794 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 295/593] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:12:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:58.420782 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b417078c-3538-481f-a3d6-e66948d54a91 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:58.422655 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084c2f9e-6839-4271-9375-6c0472316d4d None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:58.535585 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b417078c-3538-481f-a3d6-e66948d54a91 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:58.546094 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084c2f9e-6839-4271-9375-6c0472316d4d None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:58.549353 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:58.580163 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c9b55-30e5-42b7-8b00-8c095d4d9c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:58.581251 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c9b55-30e5-42b7-8b00-8c095d4d9c83 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:58.581632 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c9b55-30e5-42b7-8b00-8c095d4d9c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:58.582015 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c9b55-30e5-42b7-8b00-8c095d4d9c83 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:58.588491 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa43c5-2839-4abf-9cb5-b82fdbfd1177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:58.589294 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa43c5-2839-4abf-9cb5-b82fdbfd1177 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:58.589729 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa43c5-2839-4abf-9cb5-b82fdbfd1177 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:58.590153 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa43c5-2839-4abf-9cb5-b82fdbfd1177 None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:58.590996 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa43c5-2839-4abf-9cb5-b82fdbfd1177 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:58.591523 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa43c5-2839-4abf-9cb5-b82fdbfd1177 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:58.592035 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2c9b55-30e5-42b7-8b00-8c095d4d9c83 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:58.593319 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa43c5-2839-4abf-9cb5-b82fdbfd1177 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:58.593722 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2c9b55-30e5-42b7-8b00-8c095d4d9c83 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:58.606533 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 296/594] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:12:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:58.607538 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa43c5-2839-4abf-9cb5-b82fdbfd1177 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:58.613644 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-4f2c9b55-30e5-42b7-8b00-8c095d4d9c83 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 15:12:58.613644 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:58.613644 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:58.613644 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:58.613644 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:58.613644 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:58.613644 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:58.613644 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:58.613644 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:58.613644 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:58.613644 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:58.613644 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:58.613644 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 15:12:58.613644 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 15:12:58.613644 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 15:12:58.613644 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 15:12:58.614773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:58.614773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:58.614773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 15:12:58.614773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:12:58.614773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 15:12:58.614773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:12:58.614773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:58.614773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:58.614773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:58.614773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:58.614773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:58.614773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:58.614773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:58.614773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:58.614773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:58.614773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:58.614773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:58.614773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:58.615967 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:58.615967 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:58.615967 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 15:12:58.615967 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 15:12:58.615967 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 15:12:58.615967 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 15:12:58.615967 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 15:12:58.615967 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 15:12:58.615967 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 15:12:58.615967 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:58.622654 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 299/595] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 15:12:58 2022] GET /identity/v3/users/neutron => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:58.624059 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c9b55-30e5-42b7-8b00-8c095d4d9c83 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:58.645020 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:58.739223 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c9b55-30e5-42b7-8b00-8c095d4d9c83 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:58.740844 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa43c5-2839-4abf-9cb5-b82fdbfd1177 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:58.776609 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d485d84-7d0a-40ed-97b0-2936a1d4177e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:58.777256 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d485d84-7d0a-40ed-97b0-2936a1d4177e None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:58.777546 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d485d84-7d0a-40ed-97b0-2936a1d4177e None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:58.777837 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d485d84-7d0a-40ed-97b0-2936a1d4177e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:58.782980 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcd9d34-ace4-438e-b147-0102687b1af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:58.783622 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd9d34-ace4-438e-b147-0102687b1af1 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:58.784608 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd9d34-ace4-438e-b147-0102687b1af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:58.784949 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd9d34-ace4-438e-b147-0102687b1af1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:58.785551 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d485d84-7d0a-40ed-97b0-2936a1d4177e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:58.787000 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d485d84-7d0a-40ed-97b0-2936a1d4177e None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:58.791918 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcd9d34-ace4-438e-b147-0102687b1af1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:58.793420 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcd9d34-ace4-438e-b147-0102687b1af1 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:58.798942 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-4d485d84-7d0a-40ed-97b0-2936a1d4177e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:12:58.798942 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:58.798942 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:58.798942 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:58.798942 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:58.798942 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:58.798942 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:58.798942 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:58.798942 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:58.798942 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:58.798942 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:58.798942 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:58.798942 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 15:12:58.798942 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 15:12:58.798942 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 15:12:58.800346 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 15:12:58.800346 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:58.800346 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:58.800346 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:58.800346 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:58.800346 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:58.800346 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:58.800346 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:58.800346 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:58.800346 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:58.800346 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:58.800346 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:58.800346 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:58.800346 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:58.800346 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:58.800346 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:58.800346 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:58.801227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 15:12:58.801227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 15:12:58.801227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 15:12:58.801227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 15:12:58.801227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 15:12:58.801227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 15:12:58.801227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:12:58.801227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:58.801227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 300/596] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:12:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:58.801726 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d485d84-7d0a-40ed-97b0-2936a1d4177e None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:58.809288 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-4dcd9d34-ace4-438e-b147-0102687b1af1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:12:58.809288 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:58.809288 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:58.809288 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:58.809288 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:58.809288 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:58.809288 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:58.809288 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:58.809288 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:58.809288 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:58.809288 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:58.809288 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:58.809288 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 15:12:58.809288 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 15:12:58.809288 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 15:12:58.810162 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 15:12:58.810162 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:58.810162 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:58.810162 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:58.810162 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:58.810162 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:58.810162 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:58.810162 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:58.810162 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:58.810162 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:58.810162 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:58.810162 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:58.810162 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:58.810162 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:58.810162 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:58.810162 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:58.810162 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:58.811120 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 15:12:58.811120 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 15:12:58.811120 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 15:12:58.811120 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 15:12:58.811120 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 15:12:58.811120 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 15:12:58.811120 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:12:58.811120 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:58.812221 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 297/597] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:12:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:58.812871 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab3eab-5b5c-4412-8d2e-4cb4b18ba04f None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:58.815551 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab3eab-5b5c-4412-8d2e-4cb4b18ba04f None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:58.815551 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab3eab-5b5c-4412-8d2e-4cb4b18ba04f None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:58.815755 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 298/598] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:12:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:12:58.816345 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab3eab-5b5c-4412-8d2e-4cb4b18ba04f None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:58.847261 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:58.884014 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:58.905627 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d485d84-7d0a-40ed-97b0-2936a1d4177e None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:58.934592 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab3eab-5b5c-4412-8d2e-4cb4b18ba04f None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:58.940824 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a7d64-6d52-4a41-9e53-6dda7260c439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:58.941445 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a7d64-6d52-4a41-9e53-6dda7260c439 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:58.941743 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a7d64-6d52-4a41-9e53-6dda7260c439 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:58.942021 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a7d64-6d52-4a41-9e53-6dda7260c439 None admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:58.942857 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258a7d64-6d52-4a41-9e53-6dda7260c439 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:58.943169 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258a7d64-6d52-4a41-9e53-6dda7260c439 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:58.944626 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258a7d64-6d52-4a41-9e53-6dda7260c439 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:58.967722 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 301/599] 10.222.0.40 () {56 vars in 1139 bytes} [Fri Aug 5 15:12:58 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:58.967722 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a7d64-6d52-4a41-9e53-6dda7260c439 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:58.978383 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edd83cb-614a-4a7d-b9d1-135d8109bee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:58.979134 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd83cb-614a-4a7d-b9d1-135d8109bee2 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:58.979381 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd83cb-614a-4a7d-b9d1-135d8109bee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:58.979590 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd83cb-614a-4a7d-b9d1-135d8109bee2 None admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:58.980429 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edd83cb-614a-4a7d-b9d1-135d8109bee2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:58.980855 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edd83cb-614a-4a7d-b9d1-135d8109bee2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:58.982404 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edd83cb-614a-4a7d-b9d1-135d8109bee2 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:58.996033 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 299/600] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 15:12:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:59.002012 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1753310-3e24-49ea-9bb2-5b37a27ea59c None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:59.002307 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1753310-3e24-49ea-9bb2-5b37a27ea59c None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:59.002589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1753310-3e24-49ea-9bb2-5b37a27ea59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:59.021524 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-c1753310-3e24-49ea-9bb2-5b37a27ea59c None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:12:59.022830 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-c1753310-3e24-49ea-9bb2-5b37a27ea59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:12:59.064546 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a7d64-6d52-4a41-9e53-6dda7260c439 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:59.092783 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 300/601] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:12:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:12:59.094050 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1753310-3e24-49ea-9bb2-5b37a27ea59c None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:59.102453 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24448933-4bcf-43c0-8444-b59a9e90bfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:59.103078 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24448933-4bcf-43c0-8444-b59a9e90bfc2 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:59.103430 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24448933-4bcf-43c0-8444-b59a9e90bfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:59.103797 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24448933-4bcf-43c0-8444-b59a9e90bfc2 None admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:59.104647 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24448933-4bcf-43c0-8444-b59a9e90bfc2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:59.105023 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24448933-4bcf-43c0-8444-b59a9e90bfc2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:59.106522 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24448933-4bcf-43c0-8444-b59a9e90bfc2 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:59.109473 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:59.118225 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:59.119529 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 302/602] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 15:12:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:59.120338 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24448933-4bcf-43c0-8444-b59a9e90bfc2 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:59.218482 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1753310-3e24-49ea-9bb2-5b37a27ea59c None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:59.236793 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24448933-4bcf-43c0-8444-b59a9e90bfc2 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:59.256486 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211cb0c-3d3b-4267-8e56-4909f86c22ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:59.257136 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211cb0c-3d3b-4267-8e56-4909f86c22ef None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:59.257415 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211cb0c-3d3b-4267-8e56-4909f86c22ef None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:59.257687 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211cb0c-3d3b-4267-8e56-4909f86c22ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:59.268182 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5211cb0c-3d3b-4267-8e56-4909f86c22ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:59.270180 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5211cb0c-3d3b-4267-8e56-4909f86c22ef None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:59.275570 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b4006d-3334-4418-acc2-c03eb3127e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:59.276274 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4006d-3334-4418-acc2-c03eb3127e9b None admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:59.276561 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4006d-3334-4418-acc2-c03eb3127e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:59.276839 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4006d-3334-4418-acc2-c03eb3127e9b None admin] PATH_INFO: `/v3/projects/06f27ab8886a4c0d81c4d4e3fa100a83/users/e974f8d9a06d489884a3b00b95dd88e4/roles/266a023f66e043f99efe2941fe58d481` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:59.285848 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-5211cb0c-3d3b-4267-8e56-4909f86c22ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:59.285848 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:59.285848 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:59.285848 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:59.285848 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:59.285848 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:59.285848 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:59.285848 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:59.285848 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:59.285848 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:59.285848 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:59.285848 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:59.285848 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:12:59.285848 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:12:59.285848 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:12:59.286766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:12:59.286766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:59.286766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:59.286766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:59.286766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:59.286766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:59.286766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:59.286766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:59.286766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:59.286766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:59.286766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:59.286766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:59.286766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:59.286766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:59.286766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:59.286766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:59.286766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:59.287668 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:12:59.287668 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:12:59.287668 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:12:59.287668 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:59.287668 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 301/603] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:12:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:59.287950 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211cb0c-3d3b-4267-8e56-4909f86c22ef None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:59.296868 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b4006d-3334-4418-acc2-c03eb3127e9b None admin] RBAC: Authorizing `identity:create_grant(project_id=06f27ab8886a4c0d81c4d4e3fa100a83, user_id=e974f8d9a06d489884a3b00b95dd88e4, role_id=266a023f66e043f99efe2941fe58d481)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:59.298323 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b4006d-3334-4418-acc2-c03eb3127e9b None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:59.355825 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:59.359213 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 303/604] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 15:12:59 2022] PUT /identity/v3/projects/06f27ab8886a4c0d81c4d4e3fa100a83/users/e974f8d9a06d489884a3b00b95dd88e4/roles/266a023f66e043f99efe2941fe58d481 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:12:59.369241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b4006d-3334-4418-acc2-c03eb3127e9b None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:59.396413 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211cb0c-3d3b-4267-8e56-4909f86c22ef None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:59.438282 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb7a4b-9060-4d1e-8981-68c67e7e9164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:59.438955 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb7a4b-9060-4d1e-8981-68c67e7e9164 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:59.439238 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb7a4b-9060-4d1e-8981-68c67e7e9164 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:59.439613 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb7a4b-9060-4d1e-8981-68c67e7e9164 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:59.455221 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bb7a4b-9060-4d1e-8981-68c67e7e9164 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:12:59.461628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-b8bb7a4b-9060-4d1e-8981-68c67e7e9164 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:12:59.461628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:59.461628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:59.461628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:59.461628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:59.461628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:59.461628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:59.461628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:59.461628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:59.461628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:59.461628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:59.461628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:59.461628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:12:59.461628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:12:59.461628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:12:59.461628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:12:59.462742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:12:59.462742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:12:59.462742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:12:59.462742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:12:59.462742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:59.462742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:59.462742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:59.462742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:59.462742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:59.462742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:59.462742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:59.462742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:59.462742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:59.462742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:59.462742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:59.462742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:59.463657 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:59.463657 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:59.463657 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:59.463657 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:59.463657 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:12:59.463657 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:12:59.463657 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:12:59.463657 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:12:59.463657 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:12:59.463657 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:12:59.463657 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:12:59.463657 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:12:59.468551 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:59.470795 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 302/605] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 15:12:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:59.471819 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb7a4b-9060-4d1e-8981-68c67e7e9164 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:59.533008 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b4006d-3334-4418-acc2-c03eb3127e9b None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:59.571635 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff7079d-0583-43d8-9e10-5b25eafae2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:59.572694 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff7079d-0583-43d8-9e10-5b25eafae2ea None admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:59.573023 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff7079d-0583-43d8-9e10-5b25eafae2ea None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:59.573309 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff7079d-0583-43d8-9e10-5b25eafae2ea None admin] PATH_INFO: `/v3/projects/06f27ab8886a4c0d81c4d4e3fa100a83/users/c386fa0483cf42efaddbc2395f6d1fce/roles/266a023f66e043f99efe2941fe58d481` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:59.587072 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb7a4b-9060-4d1e-8981-68c67e7e9164 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:59.592027 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff7079d-0583-43d8-9e10-5b25eafae2ea None admin] RBAC: Authorizing `identity:create_grant(project_id=06f27ab8886a4c0d81c4d4e3fa100a83, user_id=c386fa0483cf42efaddbc2395f6d1fce, role_id=266a023f66e043f99efe2941fe58d481)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:59.593816 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff7079d-0583-43d8-9e10-5b25eafae2ea None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:59.618367 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:59.628346 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4538dc74-e538-40b1-ba6f-9e2dacbcc9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:59.629042 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538dc74-e538-40b1-ba6f-9e2dacbcc9d0 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:59.629415 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538dc74-e538-40b1-ba6f-9e2dacbcc9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:59.629758 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538dc74-e538-40b1-ba6f-9e2dacbcc9d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:59.630577 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4538dc74-e538-40b1-ba6f-9e2dacbcc9d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:59.630943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4538dc74-e538-40b1-ba6f-9e2dacbcc9d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:59.632399 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4538dc74-e538-40b1-ba6f-9e2dacbcc9d0 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:59.648106 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 303/606] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:12:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:59.648106 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4538dc74-e538-40b1-ba6f-9e2dacbcc9d0 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:59.683705 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 304/607] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 15:12:59 2022] PUT /identity/v3/projects/06f27ab8886a4c0d81c4d4e3fa100a83/users/c386fa0483cf42efaddbc2395f6d1fce/roles/266a023f66e043f99efe2941fe58d481 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:12:59.684658 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff7079d-0583-43d8-9e10-5b25eafae2ea None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:59.818225 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:12:59.826563 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff7079d-0583-43d8-9e10-5b25eafae2ea None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:59.871998 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71df80c-dab5-4e5b-89ec-7bc3bfda016a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:59.872715 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71df80c-dab5-4e5b-89ec-7bc3bfda016a None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:59.873079 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71df80c-dab5-4e5b-89ec-7bc3bfda016a None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:59.873670 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71df80c-dab5-4e5b-89ec-7bc3bfda016a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:59.885004 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71df80c-dab5-4e5b-89ec-7bc3bfda016a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:59.886484 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71df80c-dab5-4e5b-89ec-7bc3bfda016a None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:59.887335 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4538dc74-e538-40b1-ba6f-9e2dacbcc9d0 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:12:59.901655 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-c71df80c-dab5-4e5b-89ec-7bc3bfda016a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:12:59.901655 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:12:59.901655 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:12:59.901655 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:12:59.901655 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:12:59.901655 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:12:59.901655 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:12:59.901655 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:12:59.901655 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:12:59.901655 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:12:59.901655 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:12:59.901655 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:12:59.901655 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 15:12:59.901655 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 15:12:59.901655 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 15:12:59.902604 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 15:12:59.902604 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:12:59.902604 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:12:59.902604 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:12:59.902604 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:12:59.902604 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:12:59.902604 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:12:59.902604 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:12:59.902604 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:12:59.902604 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:12:59.902604 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:12:59.902604 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:12:59.902604 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:12:59.902604 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:12:59.902604 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:12:59.902604 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:12:59.902604 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:12:59.903564 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 15:12:59.903564 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 15:12:59.903564 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 15:12:59.903564 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 15:12:59.903564 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 15:12:59.903564 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 15:12:59.903564 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:12:59.903564 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:12:59.908297 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 305/608] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:12:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:12:59.911083 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71df80c-dab5-4e5b-89ec-7bc3bfda016a None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:12:59.937971 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d107dd9f-e496-4a7c-8aab-9ecf3f8c3f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:12:59.939128 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d107dd9f-e496-4a7c-8aab-9ecf3f8c3f17 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:12:59.939620 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d107dd9f-e496-4a7c-8aab-9ecf3f8c3f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:12:59.940391 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d107dd9f-e496-4a7c-8aab-9ecf3f8c3f17 None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:12:59.941576 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d107dd9f-e496-4a7c-8aab-9ecf3f8c3f17 None admin] RBAC: Adding query filter params (name=service) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:12:59.942073 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d107dd9f-e496-4a7c-8aab-9ecf3f8c3f17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:12:59.944125 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d107dd9f-e496-4a7c-8aab-9ecf3f8c3f17 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:12:59.955180 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 304/609] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 15:12:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:12:59.961242 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d107dd9f-e496-4a7c-8aab-9ecf3f8c3f17 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:00.029272 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:00.053290 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71df80c-dab5-4e5b-89ec-7bc3bfda016a None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:00.094990 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d107dd9f-e496-4a7c-8aab-9ecf3f8c3f17 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:00.095702 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ac249-13ff-4545-9a5c-6baf99893c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:00.097058 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ac249-13ff-4545-9a5c-6baf99893c38 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:00.097730 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ac249-13ff-4545-9a5c-6baf99893c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:00.098172 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ac249-13ff-4545-9a5c-6baf99893c38 None admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:00.099928 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201ac249-13ff-4545-9a5c-6baf99893c38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:00.100423 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201ac249-13ff-4545-9a5c-6baf99893c38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:00.102028 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201ac249-13ff-4545-9a5c-6baf99893c38 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:00.118754 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 306/610] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 15:12:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:00.128167 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ac249-13ff-4545-9a5c-6baf99893c38 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:00.152557 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b875d2-add2-4e25-a55d-7f341d5cc103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:00.153355 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b875d2-add2-4e25-a55d-7f341d5cc103 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:00.153743 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b875d2-add2-4e25-a55d-7f341d5cc103 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:00.154306 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b875d2-add2-4e25-a55d-7f341d5cc103 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:00.159756 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:00.181296 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b875d2-add2-4e25-a55d-7f341d5cc103 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:00.182900 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b875d2-add2-4e25-a55d-7f341d5cc103 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:00.205515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-87b875d2-add2-4e25-a55d-7f341d5cc103 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:00.205515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:00.205515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:00.205515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:00.205515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:00.205515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:00.205515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:00.205515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:00.205515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:00.205515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:00.205515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:00.205515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:00.205515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:00.205515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:00.205515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:00.207179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:00.207179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:00.207179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:00.207179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:00.207179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:00.207179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:00.207179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:00.207179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:00.207179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:00.207179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:00.207179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:00.207179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:00.207179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:00.207179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:00.207179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:00.207179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:00.207179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:00.208274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:00.208274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:00.208274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:00.208274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:00.208540 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 305/611] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:12:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:13:00.213299 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b875d2-add2-4e25-a55d-7f341d5cc103 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:00.300327 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ac249-13ff-4545-9a5c-6baf99893c38 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:00.367194 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a208c4a-6156-4cef-b853-b6d721679e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:00.367907 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a208c4a-6156-4cef-b853-b6d721679e5d None admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:00.368246 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a208c4a-6156-4cef-b853-b6d721679e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:00.368518 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a208c4a-6156-4cef-b853-b6d721679e5d None admin] PATH_INFO: `/v3/projects/06f27ab8886a4c0d81c4d4e3fa100a83/users/d13a1638b2ca497fb02c71615559d1ea/roles/266a023f66e043f99efe2941fe58d481` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:00.387240 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a208c4a-6156-4cef-b853-b6d721679e5d None admin] RBAC: Authorizing `identity:create_grant(project_id=06f27ab8886a4c0d81c4d4e3fa100a83, user_id=d13a1638b2ca497fb02c71615559d1ea, role_id=266a023f66e043f99efe2941fe58d481)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:00.388747 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a208c4a-6156-4cef-b853-b6d721679e5d None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:00.394513 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:00.397642 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b875d2-add2-4e25-a55d-7f341d5cc103 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:00.431733 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7979ee3-c912-4280-a72f-c0fc0d033867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:00.432467 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7979ee3-c912-4280-a72f-c0fc0d033867 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:00.432827 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7979ee3-c912-4280-a72f-c0fc0d033867 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:00.433181 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7979ee3-c912-4280-a72f-c0fc0d033867 None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:00.434034 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7979ee3-c912-4280-a72f-c0fc0d033867 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:00.434417 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7979ee3-c912-4280-a72f-c0fc0d033867 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:00.435884 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7979ee3-c912-4280-a72f-c0fc0d033867 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:00.452045 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 306/612] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:00.452045 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90ac58-34e2-4b6f-b30a-96a5015226b0 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:00.452045 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90ac58-34e2-4b6f-b30a-96a5015226b0 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:00.452045 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90ac58-34e2-4b6f-b30a-96a5015226b0 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:00.452045 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 307/613] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:00.454696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90ac58-34e2-4b6f-b30a-96a5015226b0 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:00.475606 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 307/614] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 15:13:00 2022] PUT /identity/v3/projects/06f27ab8886a4c0d81c4d4e3fa100a83/users/d13a1638b2ca497fb02c71615559d1ea/roles/266a023f66e043f99efe2941fe58d481 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:13:00.478392 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2b5c7-432c-4e84-8be4-c6c55a5287bd None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:00.480382 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2b5c7-432c-4e84-8be4-c6c55a5287bd None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:00.480785 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2b5c7-432c-4e84-8be4-c6c55a5287bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:00.524351 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-0cf2b5c7-432c-4e84-8be4-c6c55a5287bd None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:00.528503 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-0cf2b5c7-432c-4e84-8be4-c6c55a5287bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:00.658595 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:00.669925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 308/615] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:13:00.678475 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90ac58-34e2-4b6f-b30a-96a5015226b0 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:00.696697 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf2b5c7-432c-4e84-8be4-c6c55a5287bd None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:00.733784 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48caaeb-d067-4463-ad39-d8ff8d905b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:00.734698 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48caaeb-d067-4463-ad39-d8ff8d905b53 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:00.735129 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48caaeb-d067-4463-ad39-d8ff8d905b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:00.735540 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48caaeb-d067-4463-ad39-d8ff8d905b53 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:00.744093 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48caaeb-d067-4463-ad39-d8ff8d905b53 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:00.745749 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48caaeb-d067-4463-ad39-d8ff8d905b53 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:00.758539 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-e48caaeb-d067-4463-ad39-d8ff8d905b53 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 15:13:00.758539 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:00.758539 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:00.758539 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:00.758539 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:00.758539 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:00.758539 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:00.758539 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:00.758539 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:00.758539 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:00.758539 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:00.758539 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:00.758539 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 15:13:00.758539 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 15:13:00.758539 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 15:13:00.758539 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 15:13:00.759680 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:00.759680 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:00.759680 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 15:13:00.759680 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:00.759680 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 15:13:00.759680 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:00.759680 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:00.759680 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:00.759680 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:00.759680 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:00.759680 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:00.759680 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:00.759680 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:00.759680 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:00.759680 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:00.759680 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:00.759680 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:00.759680 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:00.760872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:00.760872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:00.760872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 15:13:00.760872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 15:13:00.760872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 15:13:00.760872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 15:13:00.760872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 15:13:00.760872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 15:13:00.760872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 15:13:00.760872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:00.763043 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 308/616] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Aug 5 15:13:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:00.768383 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48caaeb-d067-4463-ad39-d8ff8d905b53 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:00.794067 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:00.886849 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48caaeb-d067-4463-ad39-d8ff8d905b53 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:00.897224 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf2b5c7-432c-4e84-8be4-c6c55a5287bd None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:00.928537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca42ca-e126-4af7-bfd9-ba54918f8a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:00.929680 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca42ca-e126-4af7-bfd9-ba54918f8a4d None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:00.930126 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca42ca-e126-4af7-bfd9-ba54918f8a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:00.930549 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca42ca-e126-4af7-bfd9-ba54918f8a4d None admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:00.931504 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca42ca-e126-4af7-bfd9-ba54918f8a4d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:00.931965 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca42ca-e126-4af7-bfd9-ba54918f8a4d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:00.934042 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca42ca-e126-4af7-bfd9-ba54918f8a4d None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:00.954504 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 309/617] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Aug 5 15:13:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:00.965117 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca42ca-e126-4af7-bfd9-ba54918f8a4d None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:00.974124 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79493611-926c-4212-bd11-ac59be9ca011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:00.974789 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79493611-926c-4212-bd11-ac59be9ca011 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:00.975076 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79493611-926c-4212-bd11-ac59be9ca011 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:00.975356 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79493611-926c-4212-bd11-ac59be9ca011 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:00.995791 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79493611-926c-4212-bd11-ac59be9ca011 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:13:01.011929 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-79493611-926c-4212-bd11-ac59be9ca011 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:13:01.011929 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:01.011929 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:01.011929 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:01.011929 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:01.011929 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:01.011929 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:01.011929 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:01.011929 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:01.011929 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:01.011929 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:01.011929 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:01.011929 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:13:01.011929 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:13:01.011929 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:13:01.011929 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:13:01.013136 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:13:01.013136 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:13:01.013136 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:13:01.013136 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:13:01.013136 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:01.013136 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:01.013136 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:01.013136 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:01.013136 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:01.013136 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:01.013136 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:01.013136 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:01.013136 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:01.013136 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:01.013136 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:01.013136 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:01.014162 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:01.014162 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:01.014162 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:01.014162 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:01.014162 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:13:01.014162 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:13:01.014162 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:13:01.014162 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:13:01.014162 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:13:01.014162 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:13:01.014162 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:13:01.014162 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:01.014162 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 309/618] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 15:13:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:13:01.022394 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79493611-926c-4212-bd11-ac59be9ca011 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:01.060228 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:01.084209 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca42ca-e126-4af7-bfd9-ba54918f8a4d None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:01.119850 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:01.121891 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fcade6-d6c4-453e-8956-6481a0fd8ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:01.122686 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fcade6-d6c4-453e-8956-6481a0fd8ba9 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:01.123117 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fcade6-d6c4-453e-8956-6481a0fd8ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:01.123814 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fcade6-d6c4-453e-8956-6481a0fd8ba9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:01.199685 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79493611-926c-4212-bd11-ac59be9ca011 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:01.201760 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fcade6-d6c4-453e-8956-6481a0fd8ba9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:01.203377 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fcade6-d6c4-453e-8956-6481a0fd8ba9 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:01.265709 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2828d0df-0965-4cee-8fa4-3b1f401e3e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:01.266510 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2828d0df-0965-4cee-8fa4-3b1f401e3e06 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:01.267083 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2828d0df-0965-4cee-8fa4-3b1f401e3e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:01.267258 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2828d0df-0965-4cee-8fa4-3b1f401e3e06 None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:01.268625 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2828d0df-0965-4cee-8fa4-3b1f401e3e06 None admin] RBAC: Adding query filter params (name=service) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:01.269043 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2828d0df-0965-4cee-8fa4-3b1f401e3e06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:01.270506 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2828d0df-0965-4cee-8fa4-3b1f401e3e06 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:01.289103 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:01.294688 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 310/619] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 15:13:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:01.313219 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2828d0df-0965-4cee-8fa4-3b1f401e3e06 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:01.318943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 310/620] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:13:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 15:13:01.329007 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fcade6-d6c4-453e-8956-6481a0fd8ba9 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:01.430487 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fcade6-d6c4-453e-8956-6481a0fd8ba9 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:01.467324 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc65d32-53c3-4a5a-afa5-6205f2bddc27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:01.468236 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc65d32-53c3-4a5a-afa5-6205f2bddc27 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:01.468666 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc65d32-53c3-4a5a-afa5-6205f2bddc27 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:01.469112 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc65d32-53c3-4a5a-afa5-6205f2bddc27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:01.479653 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc65d32-53c3-4a5a-afa5-6205f2bddc27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:01.481333 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc65d32-53c3-4a5a-afa5-6205f2bddc27 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:01.496517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-1dc65d32-53c3-4a5a-afa5-6205f2bddc27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:01.496517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:01.496517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:01.496517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:01.496517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:01.496517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:01.496517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:01.496517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:01.496517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:01.496517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:01.496517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:01.496517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:01.496517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:01.496517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:01.496517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:01.497589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:01.497589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:01.497589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:01.497589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:01.497589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:01.497589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:01.497589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:01.497589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:01.497589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:01.497589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:01.497589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:01.497589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:01.497589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:01.497589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:01.497589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:01.497589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:01.497589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:01.498917 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:01.498917 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:01.498917 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:01.498917 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:01.500734 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 311/621] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:01.506515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc65d32-53c3-4a5a-afa5-6205f2bddc27 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:01.517207 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2828d0df-0965-4cee-8fa4-3b1f401e3e06 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:01.556508 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:01.557017 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:01.618099 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3512859-1a0a-4614-8903-44caf3b6be32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:01.618759 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3512859-1a0a-4614-8903-44caf3b6be32 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:01.619053 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3512859-1a0a-4614-8903-44caf3b6be32 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:01.619334 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3512859-1a0a-4614-8903-44caf3b6be32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:01.628952 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc65d32-53c3-4a5a-afa5-6205f2bddc27 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:01.647252 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3512859-1a0a-4614-8903-44caf3b6be32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:01.649447 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3512859-1a0a-4614-8903-44caf3b6be32 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:01.666230 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f39136-9e84-4d04-98ae-f95c50ca7b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:01.666860 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f39136-9e84-4d04-98ae-f95c50ca7b7c None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:01.667145 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f39136-9e84-4d04-98ae-f95c50ca7b7c None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:01.667430 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f39136-9e84-4d04-98ae-f95c50ca7b7c None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:01.668596 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f39136-9e84-4d04-98ae-f95c50ca7b7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:01.668918 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f39136-9e84-4d04-98ae-f95c50ca7b7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:01.670305 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f39136-9e84-4d04-98ae-f95c50ca7b7c None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:01.683352 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 312/622] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:01.688458 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-e3512859-1a0a-4614-8903-44caf3b6be32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:01.688458 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:01.688458 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:01.688458 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:01.688458 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:01.688458 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:01.688458 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:01.688458 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:01.688458 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:01.688458 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:01.688458 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:01.688458 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:01.688458 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:01.688458 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:01.688458 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:01.689306 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:01.689306 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:01.689306 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:01.689306 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:01.689306 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:01.689306 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:01.689306 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:01.689306 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:01.689306 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:01.689306 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:01.689306 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:01.689306 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:01.689306 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:01.689306 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:01.689306 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:01.689306 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:01.689306 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:01.690719 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:01.690719 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:01.690719 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:01.690719 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:01.694689 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 311/623] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:01.696208 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f39136-9e84-4d04-98ae-f95c50ca7b7c None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:01.697467 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3512859-1a0a-4614-8903-44caf3b6be32 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:01.803523 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:01.813826 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f39136-9e84-4d04-98ae-f95c50ca7b7c None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:01.847295 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3512859-1a0a-4614-8903-44caf3b6be32 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:01.860497 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e773bd3-fa1d-428d-becb-5838cb3a2400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:01.862349 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e773bd3-fa1d-428d-becb-5838cb3a2400 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:01.862774 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e773bd3-fa1d-428d-becb-5838cb3a2400 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:01.863536 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e773bd3-fa1d-428d-becb-5838cb3a2400 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:01.872646 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e773bd3-fa1d-428d-becb-5838cb3a2400 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:01.875193 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e773bd3-fa1d-428d-becb-5838cb3a2400 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:01.885178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa603a3-021a-41a7-8e73-e8a1730e03fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:01.885953 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa603a3-021a-41a7-8e73-e8a1730e03fa None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:01.886306 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa603a3-021a-41a7-8e73-e8a1730e03fa None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:01.886702 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa603a3-021a-41a7-8e73-e8a1730e03fa None admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:01.887467 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa603a3-021a-41a7-8e73-e8a1730e03fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:01.887848 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa603a3-021a-41a7-8e73-e8a1730e03fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:01.889436 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa603a3-021a-41a7-8e73-e8a1730e03fa None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:01.890444 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-8e773bd3-fa1d-428d-becb-5838cb3a2400 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:01.890444 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:01.890444 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:01.890444 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:01.890444 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:01.890444 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:01.890444 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:01.890444 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:01.890444 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:01.890444 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:01.890444 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:01.890444 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:01.890444 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 15:13:01.890444 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 15:13:01.890444 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 15:13:01.891511 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 15:13:01.891511 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:01.891511 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:01.891511 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:01.891511 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:01.891511 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:01.891511 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:01.891511 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:01.891511 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:01.891511 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:01.891511 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:01.891511 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:01.891511 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:01.891511 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:01.891511 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:01.891511 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:01.891511 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:01.892823 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 15:13:01.892823 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 15:13:01.892823 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 15:13:01.892823 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 15:13:01.892823 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 15:13:01.892823 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 15:13:01.892823 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:01.892823 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:01.900696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 313/624] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:13:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:01.903104 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd68db5a-43fe-4670-b8fb-9204f04507eb None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:01.903608 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd68db5a-43fe-4670-b8fb-9204f04507eb None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:01.904348 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd68db5a-43fe-4670-b8fb-9204f04507eb None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:01.906454 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 314/625] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:01.906873 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 312/626] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:01.907856 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd68db5a-43fe-4670-b8fb-9204f04507eb None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:01.911563 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675f9be-93b6-4f1e-90ac-e15bb273d935 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:01.912108 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675f9be-93b6-4f1e-90ac-e15bb273d935 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:01.912625 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675f9be-93b6-4f1e-90ac-e15bb273d935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:01.932090 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-a675f9be-93b6-4f1e-90ac-e15bb273d935 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:01.933797 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-a675f9be-93b6-4f1e-90ac-e15bb273d935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:01.960905 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:02.020230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 313/627] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:13:02.021183 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a675f9be-93b6-4f1e-90ac-e15bb273d935 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:02.039821 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd68db5a-43fe-4670-b8fb-9204f04507eb None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:02.083621 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb6e7b-ba83-4b96-919d-84018ea52f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:02.085173 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb6e7b-ba83-4b96-919d-84018ea52f55 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:02.085665 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb6e7b-ba83-4b96-919d-84018ea52f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:02.086245 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb6e7b-ba83-4b96-919d-84018ea52f55 None admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:02.087198 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbb6e7b-ba83-4b96-919d-84018ea52f55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:02.087712 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbb6e7b-ba83-4b96-919d-84018ea52f55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:02.089664 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbb6e7b-ba83-4b96-919d-84018ea52f55 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:02.108097 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 315/628] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 15:13:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:02.109609 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb6e7b-ba83-4b96-919d-84018ea52f55 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:02.124192 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:02.143994 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a675f9be-93b6-4f1e-90ac-e15bb273d935 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:02.188711 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2d2e7-28d2-4aae-bade-83cf1182c222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:02.189672 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2d2e7-28d2-4aae-bade-83cf1182c222 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:02.190417 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2d2e7-28d2-4aae-bade-83cf1182c222 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:02.190851 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2d2e7-28d2-4aae-bade-83cf1182c222 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:02.200768 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b2d2e7-28d2-4aae-bade-83cf1182c222 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:02.205034 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b2d2e7-28d2-4aae-bade-83cf1182c222 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:02.219349 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-c6b2d2e7-28d2-4aae-bade-83cf1182c222 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 15:13:02.219349 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:02.219349 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:02.219349 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:02.219349 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:02.219349 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:02.219349 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:02.219349 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:02.219349 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:02.219349 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:02.219349 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:02.219349 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:02.219349 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 15:13:02.219349 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 15:13:02.219349 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 15:13:02.219349 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 15:13:02.220495 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:02.220495 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:02.220495 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 15:13:02.220495 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:02.220495 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 15:13:02.220495 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:02.220495 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:02.220495 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:02.220495 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:02.220495 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:02.220495 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:02.220495 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:02.220495 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:02.220495 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:02.220495 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:02.220495 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:02.220495 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:02.220495 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:02.222018 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:02.222018 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:02.222018 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 15:13:02.222018 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 15:13:02.222018 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 15:13:02.222018 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 15:13:02.222018 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 15:13:02.222018 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 15:13:02.222018 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 15:13:02.222018 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:02.222018 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 314/629] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 15:13:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:02.222018 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2d2e7-28d2-4aae-bade-83cf1182c222 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:02.238549 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb6e7b-ba83-4b96-919d-84018ea52f55 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:02.247242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:02.274979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2bd276-27a0-4b3e-8620-bc5a9eaf82ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:02.275621 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2bd276-27a0-4b3e-8620-bc5a9eaf82ca None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:02.275904 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2bd276-27a0-4b3e-8620-bc5a9eaf82ca None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:02.276224 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2bd276-27a0-4b3e-8620-bc5a9eaf82ca None admin] PATH_INFO: `/v3/roles/service` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:02.286980 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2bd276-27a0-4b3e-8620-bc5a9eaf82ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:13:02.291791 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-cf2bd276-27a0-4b3e-8620-bc5a9eaf82ca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:13:02.291791 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:02.291791 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:02.291791 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:02.291791 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:02.291791 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:02.291791 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:02.291791 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:02.291791 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:02.291791 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:02.291791 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:02.291791 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:02.291791 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:13:02.291791 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:13:02.291791 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:13:02.291791 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:13:02.292962 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:13:02.292962 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:13:02.292962 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:13:02.292962 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:13:02.292962 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:02.292962 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:02.292962 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:02.292962 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:02.292962 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:02.292962 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:02.292962 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:02.292962 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:02.292962 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:02.292962 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:02.292962 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:02.292962 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:02.293819 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:02.293819 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:02.293819 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:02.293819 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:02.293819 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:13:02.293819 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:13:02.293819 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:13:02.293819 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:13:02.293819 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:13:02.293819 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:13:02.293819 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:13:02.293819 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:02.293819 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 316/630] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 15:13:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:02.294561 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a04f0-00f2-4fc2-91ef-b166ffbd9b50 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:02.294667 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a04f0-00f2-4fc2-91ef-b166ffbd9b50 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:02.294943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a04f0-00f2-4fc2-91ef-b166ffbd9b50 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:02.300782 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 317/631] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:02.301421 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a04f0-00f2-4fc2-91ef-b166ffbd9b50 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:02.344375 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2d2e7-28d2-4aae-bade-83cf1182c222 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:02.375407 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:02.381913 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81805afa-6c7f-4de4-800e-175b2178a21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:02.382536 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81805afa-6c7f-4de4-800e-175b2178a21b None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:02.382816 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81805afa-6c7f-4de4-800e-175b2178a21b None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:02.383095 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81805afa-6c7f-4de4-800e-175b2178a21b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:02.383886 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81805afa-6c7f-4de4-800e-175b2178a21b None admin] RBAC: Adding query filter params (role.id=266a023f66e043f99efe2941fe58d481, scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83, user.id=bc0497dab1f44bb7b8196d39bf01d770) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:02.384293 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81805afa-6c7f-4de4-800e-175b2178a21b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:02.388216 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81805afa-6c7f-4de4-800e-175b2178a21b None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:02.391866 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 315/632] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:13:02 2022] GET /identity/v3/role_assignments?user.id=bc0497dab1f44bb7b8196d39bf01d770&scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83&role.id=266a023f66e043f99efe2941fe58d481 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:02.392441 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81805afa-6c7f-4de4-800e-175b2178a21b None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:02.419472 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a04f0-00f2-4fc2-91ef-b166ffbd9b50 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:02.459039 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322e758a-1319-44db-b12c-43cb068fc727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:02.459679 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e758a-1319-44db-b12c-43cb068fc727 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:02.459959 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e758a-1319-44db-b12c-43cb068fc727 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:02.460269 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e758a-1319-44db-b12c-43cb068fc727 None admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:02.461151 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322e758a-1319-44db-b12c-43cb068fc727 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:02.461458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322e758a-1319-44db-b12c-43cb068fc727 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:02.463045 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322e758a-1319-44db-b12c-43cb068fc727 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:02.488510 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 318/633] 10.222.0.40 () {56 vars in 1135 bytes} [Fri Aug 5 15:13:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:02.489929 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc53e6-06b4-47be-9c94-3ab625a91c2c None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:02.490222 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc53e6-06b4-47be-9c94-3ab625a91c2c None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:02.490501 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc53e6-06b4-47be-9c94-3ab625a91c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:02.510471 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-00fc53e6-06b4-47be-9c94-3ab625a91c2c None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:02.511707 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-00fc53e6-06b4-47be-9c94-3ab625a91c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:02.521619 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:02.544770 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81805afa-6c7f-4de4-800e-175b2178a21b None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:02.592017 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a31911-cc41-4321-9442-11c51419ba1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:02.592683 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a31911-cc41-4321-9442-11c51419ba1f None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:02.592973 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a31911-cc41-4321-9442-11c51419ba1f None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:02.593256 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a31911-cc41-4321-9442-11c51419ba1f None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:02.594055 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a31911-cc41-4321-9442-11c51419ba1f None admin] RBAC: Adding query filter params (name=service) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:02.594374 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a31911-cc41-4321-9442-11c51419ba1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:02.595809 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a31911-cc41-4321-9442-11c51419ba1f None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:02.597856 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 319/634] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:13:02.599038 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc53e6-06b4-47be-9c94-3ab625a91c2c None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:02.610148 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 316/635] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 15:13:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:02.611534 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a31911-cc41-4321-9442-11c51419ba1f None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:02.681928 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:02.735590 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc53e6-06b4-47be-9c94-3ab625a91c2c None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:02.736651 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a31911-cc41-4321-9442-11c51419ba1f None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:02.772905 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ddf6f-7e5b-4938-a1d6-3fb71eb6d8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:02.773585 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ddf6f-7e5b-4938-a1d6-3fb71eb6d8fe None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:02.773871 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ddf6f-7e5b-4938-a1d6-3fb71eb6d8fe None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:02.774152 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ddf6f-7e5b-4938-a1d6-3fb71eb6d8fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:02.785079 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c6456d-c88e-45cf-82a7-bce9dfdf430a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:02.786342 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6456d-c88e-45cf-82a7-bce9dfdf430a None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:02.786713 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3ddf6f-7e5b-4938-a1d6-3fb71eb6d8fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:02.787072 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6456d-c88e-45cf-82a7-bce9dfdf430a None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:02.787558 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6456d-c88e-45cf-82a7-bce9dfdf430a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:02.788223 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3ddf6f-7e5b-4938-a1d6-3fb71eb6d8fe None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:02.795642 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:02.802131 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c6456d-c88e-45cf-82a7-bce9dfdf430a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:02.803902 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c6456d-c88e-45cf-82a7-bce9dfdf430a None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:02.804872 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-3a3ddf6f-7e5b-4938-a1d6-3fb71eb6d8fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:02.804872 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:02.804872 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:02.804872 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:02.804872 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:02.804872 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:02.804872 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:02.804872 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:02.804872 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:02.804872 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:02.804872 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:02.804872 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:02.804872 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:02.804872 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:02.804872 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:02.811372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:02.811372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:02.811372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:02.811372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:02.811372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:02.811372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:02.811372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:02.811372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:02.811372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:02.811372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:02.811372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:02.811372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:02.811372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:02.811372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:02.811372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:02.811372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:02.811372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:02.812362 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:02.812362 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:02.812362 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:02.812362 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:02.812362 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 317/636] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:13:02.812362 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ddf6f-7e5b-4938-a1d6-3fb71eb6d8fe None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:02.831067 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-19c6456d-c88e-45cf-82a7-bce9dfdf430a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:02.831067 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:02.831067 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:02.831067 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:02.831067 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:02.831067 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:02.831067 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:02.831067 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:02.831067 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:02.831067 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:02.831067 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:02.831067 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:02.831067 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:02.831067 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:02.831067 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:02.832024 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:02.832024 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:02.832024 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:02.832024 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:02.832024 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:02.832024 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:02.832024 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:02.832024 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:02.832024 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:02.832024 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:02.832024 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:02.832024 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:02.832024 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:02.832024 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:02.832024 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:02.832024 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:02.832024 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:02.832945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:02.832945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:02.832945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:02.832945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:02.832945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 320/637] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:02.836088 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c6456d-c88e-45cf-82a7-bce9dfdf430a None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:02.922504 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ddf6f-7e5b-4938-a1d6-3fb71eb6d8fe None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:02.956578 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c6456d-c88e-45cf-82a7-bce9dfdf430a None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:02.959985 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b776ad-ab4d-4db5-ad8c-3fce9c3e3995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:02.960672 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b776ad-ab4d-4db5-ad8c-3fce9c3e3995 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:02.960953 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b776ad-ab4d-4db5-ad8c-3fce9c3e3995 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:02.961228 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b776ad-ab4d-4db5-ad8c-3fce9c3e3995 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:02.969684 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:02.972583 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b776ad-ab4d-4db5-ad8c-3fce9c3e3995 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:13:02.978038 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-49b776ad-ab4d-4db5-ad8c-3fce9c3e3995 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:13:02.978038 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:02.978038 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:02.978038 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:02.978038 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:02.978038 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:02.978038 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:02.978038 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:02.978038 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:02.978038 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:02.978038 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:02.978038 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:02.978038 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:13:02.978038 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:13:02.978038 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:13:02.978038 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:13:02.979041 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:13:02.979041 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:13:02.979041 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:13:02.979041 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:13:02.979041 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:02.979041 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:02.979041 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:02.979041 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:02.979041 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:02.979041 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:02.979041 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:02.979041 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:02.979041 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:02.979041 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:02.979041 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:02.979041 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:02.979908 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:02.979908 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:02.979908 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:02.979908 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:02.979908 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:13:02.979908 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:13:02.979908 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:13:02.979908 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:13:02.979908 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:13:02.979908 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:13:02.979908 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:13:02.979908 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:02.979908 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 318/638] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 15:13:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:02.979908 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b776ad-ab4d-4db5-ad8c-3fce9c3e3995 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:03.004263 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10b1cf6-b12e-49e5-8119-906afe50846b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:03.004987 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10b1cf6-b12e-49e5-8119-906afe50846b None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:03.005277 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10b1cf6-b12e-49e5-8119-906afe50846b None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:03.005559 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10b1cf6-b12e-49e5-8119-906afe50846b None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:03.006257 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10b1cf6-b12e-49e5-8119-906afe50846b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:03.006573 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10b1cf6-b12e-49e5-8119-906afe50846b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:03.007983 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10b1cf6-b12e-49e5-8119-906afe50846b None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:03.021615 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 321/639] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:03.027024 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c4e25-1523-4903-9700-4b6b7384557b None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:03.027317 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c4e25-1523-4903-9700-4b6b7384557b None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:03.027600 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c4e25-1523-4903-9700-4b6b7384557b None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:03.028882 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 322/640] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:03.029583 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c4e25-1523-4903-9700-4b6b7384557b None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:03.050018 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:03.095776 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b776ad-ab4d-4db5-ad8c-3fce9c3e3995 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:03.130988 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a70ef55-686b-44a4-84f6-7a63522b714a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:03.131609 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70ef55-686b-44a4-84f6-7a63522b714a None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:03.131901 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70ef55-686b-44a4-84f6-7a63522b714a None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:03.132215 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70ef55-686b-44a4-84f6-7a63522b714a None admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:03.132907 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a70ef55-686b-44a4-84f6-7a63522b714a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:03.133245 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a70ef55-686b-44a4-84f6-7a63522b714a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:03.134345 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c4e25-1523-4903-9700-4b6b7384557b None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:03.134879 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a70ef55-686b-44a4-84f6-7a63522b714a None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:03.151979 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 319/641] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:03.151979 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a70ef55-686b-44a4-84f6-7a63522b714a None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:03.174677 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31885c4a-6916-461b-a341-9e6d9c965f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:03.175248 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31885c4a-6916-461b-a341-9e6d9c965f04 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:03.175535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31885c4a-6916-461b-a341-9e6d9c965f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:03.175816 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31885c4a-6916-461b-a341-9e6d9c965f04 None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:03.176569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31885c4a-6916-461b-a341-9e6d9c965f04 None admin] RBAC: Adding query filter params (name=service) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:03.176878 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31885c4a-6916-461b-a341-9e6d9c965f04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:03.178268 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31885c4a-6916-461b-a341-9e6d9c965f04 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:03.186533 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 323/642] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 15:13:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:03.187554 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ca295-d437-467e-87ae-f72842276773 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:03.187848 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ca295-d437-467e-87ae-f72842276773 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:03.188169 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ca295-d437-467e-87ae-f72842276773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:03.206899 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-2f8ca295-d437-467e-87ae-f72842276773 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:03.208118 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-2f8ca295-d437-467e-87ae-f72842276773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:03.235383 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:03.272399 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a70ef55-686b-44a4-84f6-7a63522b714a None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:03.280482 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 324/643] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:13:03.281326 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ca295-d437-467e-87ae-f72842276773 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:03.304271 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:03.323041 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f9202-a19d-4bb0-970f-36600793e5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:03.323688 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f9202-a19d-4bb0-970f-36600793e5e3 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:03.323978 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f9202-a19d-4bb0-970f-36600793e5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:03.324299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f9202-a19d-4bb0-970f-36600793e5e3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:03.332249 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f9202-a19d-4bb0-970f-36600793e5e3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:03.333823 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f9202-a19d-4bb0-970f-36600793e5e3 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:03.345768 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-e18f9202-a19d-4bb0-970f-36600793e5e3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 15:13:03.345768 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:03.345768 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:03.345768 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:03.345768 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:03.345768 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:03.345768 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:03.345768 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:03.345768 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:03.345768 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:03.345768 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:03.345768 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:03.345768 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 15:13:03.345768 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 15:13:03.345768 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 15:13:03.345768 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 15:13:03.346925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:03.346925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:03.346925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 15:13:03.346925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:03.346925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 15:13:03.346925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:03.346925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:03.346925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:03.346925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:03.346925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:03.346925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:03.346925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:03.346925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:03.346925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:03.346925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:03.346925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:03.346925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:03.346925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:03.347875 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:03.347875 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:03.347875 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 15:13:03.347875 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 15:13:03.347875 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 15:13:03.347875 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 15:13:03.347875 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 15:13:03.347875 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 15:13:03.347875 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 15:13:03.347875 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:03.347875 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 320/644] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:13:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:03.348541 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f9202-a19d-4bb0-970f-36600793e5e3 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:03.391729 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ca295-d437-467e-87ae-f72842276773 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:03.426520 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9330a15e-d636-4ea1-ac0a-9eb8e7fca4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:03.427234 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330a15e-d636-4ea1-ac0a-9eb8e7fca4e8 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:03.427592 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330a15e-d636-4ea1-ac0a-9eb8e7fca4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:03.427939 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330a15e-d636-4ea1-ac0a-9eb8e7fca4e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:03.434895 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9330a15e-d636-4ea1-ac0a-9eb8e7fca4e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:03.436448 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9330a15e-d636-4ea1-ac0a-9eb8e7fca4e8 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:03.448626 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-9330a15e-d636-4ea1-ac0a-9eb8e7fca4e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:03.448626 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:03.448626 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:03.448626 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:03.448626 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:03.448626 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:03.448626 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:03.448626 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:03.448626 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:03.448626 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:03.448626 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:03.448626 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:03.448626 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 15:13:03.448626 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 15:13:03.448626 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 15:13:03.449693 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 15:13:03.449693 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:03.449693 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:03.449693 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:03.449693 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:03.449693 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:03.449693 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:03.449693 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:03.449693 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:03.449693 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:03.449693 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:03.449693 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:03.449693 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:03.449693 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:03.449693 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:03.449693 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:03.449693 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:03.450887 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 15:13:03.450887 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 15:13:03.450887 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 15:13:03.450887 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 15:13:03.450887 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 15:13:03.450887 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 15:13:03.450887 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:03.450887 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:03.450887 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 325/645] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:13:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:03.455219 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9330a15e-d636-4ea1-ac0a-9eb8e7fca4e8 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:03.460974 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f9202-a19d-4bb0-970f-36600793e5e3 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:03.481701 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:03.495920 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3997c8d3-e7e2-4dec-ab48-314eecbc2ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:03.497188 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3997c8d3-e7e2-4dec-ab48-314eecbc2ba7 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:03.497188 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3997c8d3-e7e2-4dec-ab48-314eecbc2ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:03.497394 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3997c8d3-e7e2-4dec-ab48-314eecbc2ba7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:03.509160 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3997c8d3-e7e2-4dec-ab48-314eecbc2ba7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:03.510737 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3997c8d3-e7e2-4dec-ab48-314eecbc2ba7 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:03.525867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-3997c8d3-e7e2-4dec-ab48-314eecbc2ba7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:03.525867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:03.525867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:03.525867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:03.525867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:03.525867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:03.525867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:03.525867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:03.525867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:03.525867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:03.525867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:03.525867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:03.525867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:03.525867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:03.525867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:03.526915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:03.526915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:03.526915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:03.526915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:03.526915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:03.526915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:03.526915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:03.526915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:03.526915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:03.526915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:03.526915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:03.526915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:03.526915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:03.526915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:03.526915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:03.526915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:03.526915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:03.527985 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:03.527985 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:03.527985 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:03.527985 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:03.533155 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 321/646] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:03.533155 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3997c8d3-e7e2-4dec-ab48-314eecbc2ba7 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:03.563447 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:03.570129 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9330a15e-d636-4ea1-ac0a-9eb8e7fca4e8 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:03.604023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4cd5c8-8ce1-4667-86ca-ff5f36405c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:03.604847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4cd5c8-8ce1-4667-86ca-ff5f36405c59 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:03.605235 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4cd5c8-8ce1-4667-86ca-ff5f36405c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:03.605609 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4cd5c8-8ce1-4667-86ca-ff5f36405c59 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:03.616584 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4cd5c8-8ce1-4667-86ca-ff5f36405c59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:13:03.621584 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-fa4cd5c8-8ce1-4667-86ca-ff5f36405c59 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:13:03.621584 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:03.621584 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:03.621584 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:03.621584 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:03.621584 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:03.621584 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:03.621584 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:03.621584 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:03.621584 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:03.621584 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:03.621584 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:03.621584 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:13:03.621584 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:13:03.621584 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:13:03.621584 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:13:03.622705 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:13:03.622705 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:13:03.622705 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:13:03.622705 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:13:03.622705 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:03.622705 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:03.622705 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:03.622705 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:03.622705 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:03.622705 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:03.622705 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:03.622705 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:03.622705 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:03.622705 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:03.622705 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:03.622705 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:03.624028 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:03.624028 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:03.624028 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:03.624028 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:03.624028 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:13:03.624028 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:13:03.624028 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:13:03.624028 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:13:03.624028 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:13:03.624028 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:13:03.624028 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:13:03.624028 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:03.625638 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 326/647] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 15:13:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:03.626477 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4cd5c8-8ce1-4667-86ca-ff5f36405c59 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:03.644241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3997c8d3-e7e2-4dec-ab48-314eecbc2ba7 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:03.682275 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122216b5-e4e3-4892-8d41-43f0841f57c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:03.682813 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122216b5-e4e3-4892-8d41-43f0841f57c5 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:03.683098 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122216b5-e4e3-4892-8d41-43f0841f57c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:03.683379 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122216b5-e4e3-4892-8d41-43f0841f57c5 None admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:03.684205 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122216b5-e4e3-4892-8d41-43f0841f57c5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:03.684520 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122216b5-e4e3-4892-8d41-43f0841f57c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:03.685955 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122216b5-e4e3-4892-8d41-43f0841f57c5 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:03.703493 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 322/648] 10.222.0.40 () {56 vars in 1137 bytes} [Fri Aug 5 15:13:03 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:03.704335 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122216b5-e4e3-4892-8d41-43f0841f57c5 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:03.720661 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:03.743913 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4cd5c8-8ce1-4667-86ca-ff5f36405c59 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:03.789585 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569338db-b963-450a-a49e-50c086120cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:03.790207 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569338db-b963-450a-a49e-50c086120cef None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:03.790647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569338db-b963-450a-a49e-50c086120cef None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:03.790933 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569338db-b963-450a-a49e-50c086120cef None admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:03.791976 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569338db-b963-450a-a49e-50c086120cef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:03.792321 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569338db-b963-450a-a49e-50c086120cef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:03.793765 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569338db-b963-450a-a49e-50c086120cef None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:03.794219 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:03.806709 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 327/649] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 15:13:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:03.810610 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122216b5-e4e3-4892-8d41-43f0841f57c5 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:03.812311 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569338db-b963-450a-a49e-50c086120cef None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:03.849507 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b457ac4-92cf-4f36-84d0-80907e62996a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:03.850123 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b457ac4-92cf-4f36-84d0-80907e62996a None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:03.850403 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b457ac4-92cf-4f36-84d0-80907e62996a None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:03.850676 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b457ac4-92cf-4f36-84d0-80907e62996a None admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:03.851361 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b457ac4-92cf-4f36-84d0-80907e62996a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:03.851667 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b457ac4-92cf-4f36-84d0-80907e62996a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:03.853058 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b457ac4-92cf-4f36-84d0-80907e62996a None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:03.872504 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 323/650] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:03.872504 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b457ac4-92cf-4f36-84d0-80907e62996a None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:03.915994 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569338db-b963-450a-a49e-50c086120cef None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:03.949733 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc36e909-2f79-4513-95bb-b70fad1f64b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:03.950283 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36e909-2f79-4513-95bb-b70fad1f64b5 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:03.950573 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36e909-2f79-4513-95bb-b70fad1f64b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:03.950853 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36e909-2f79-4513-95bb-b70fad1f64b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:03.951574 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc36e909-2f79-4513-95bb-b70fad1f64b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:03.951892 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc36e909-2f79-4513-95bb-b70fad1f64b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:03.953312 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc36e909-2f79-4513-95bb-b70fad1f64b5 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:03.965668 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 328/651] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 15:13:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:03.965668 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc36e909-2f79-4513-95bb-b70fad1f64b5 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:03.967353 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:03.986511 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b457ac4-92cf-4f36-84d0-80907e62996a None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:04.025998 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03a4e21-890d-4e87-afa5-df4cad906a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:04.026731 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a4e21-890d-4e87-afa5-df4cad906a57 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:04.027014 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a4e21-890d-4e87-afa5-df4cad906a57 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:04.027295 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a4e21-890d-4e87-afa5-df4cad906a57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:04.046407 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:04.071359 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc36e909-2f79-4513-95bb-b70fad1f64b5 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:04.100275 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03a4e21-890d-4e87-afa5-df4cad906a57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:04.101727 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03a4e21-890d-4e87-afa5-df4cad906a57 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:04.108098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f561882-fb8f-4c6f-9b9e-bb2619bda5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:04.108775 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f561882-fb8f-4c6f-9b9e-bb2619bda5e7 None admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:04.109100 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f561882-fb8f-4c6f-9b9e-bb2619bda5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:04.109332 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f561882-fb8f-4c6f-9b9e-bb2619bda5e7 None admin] PATH_INFO: `/v3/projects/06f27ab8886a4c0d81c4d4e3fa100a83/users/85154ce7446245738edcc8702dbf6861/roles/266a023f66e043f99efe2941fe58d481` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:04.125417 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f561882-fb8f-4c6f-9b9e-bb2619bda5e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=06f27ab8886a4c0d81c4d4e3fa100a83, user_id=85154ce7446245738edcc8702dbf6861, role_id=266a023f66e043f99efe2941fe58d481)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:04.126867 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f561882-fb8f-4c6f-9b9e-bb2619bda5e7 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:04.186711 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 329/652] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 15:13:03 2022] PUT /identity/v3/projects/06f27ab8886a4c0d81c4d4e3fa100a83/users/85154ce7446245738edcc8702dbf6861/roles/266a023f66e043f99efe2941fe58d481 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:13:04.197121 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f561882-fb8f-4c6f-9b9e-bb2619bda5e7 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:04.216620 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 324/653] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:13:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 15:13:04.216620 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03a4e21-890d-4e87-afa5-df4cad906a57 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:04.271189 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:04.286764 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:04.327914 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03a4e21-890d-4e87-afa5-df4cad906a57 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:04.337132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f561882-fb8f-4c6f-9b9e-bb2619bda5e7 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:04.366214 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b29c0-c1c8-4c8a-b344-d73b6a6e2bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:04.366850 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b29c0-c1c8-4c8a-b344-d73b6a6e2bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:04.367133 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b29c0-c1c8-4c8a-b344-d73b6a6e2bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:04.367406 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b29c0-c1c8-4c8a-b344-d73b6a6e2bc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:04.377111 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0586d5e2-b6c6-468d-822a-60a0c5cd3b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:04.377775 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0586d5e2-b6c6-468d-822a-60a0c5cd3b3e None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:04.378116 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0586d5e2-b6c6-468d-822a-60a0c5cd3b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:04.378442 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0586d5e2-b6c6-468d-822a-60a0c5cd3b3e None admin] PATH_INFO: `/v3/users/glance` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:04.380434 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b29c0-c1c8-4c8a-b344-d73b6a6e2bc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:04.382049 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b29c0-c1c8-4c8a-b344-d73b6a6e2bc8 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:04.387874 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0586d5e2-b6c6-468d-822a-60a0c5cd3b3e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:04.390108 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0586d5e2-b6c6-468d-822a-60a0c5cd3b3e None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:04.404159 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-0e7b29c0-c1c8-4c8a-b344-d73b6a6e2bc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:04.404159 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:04.404159 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:04.404159 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:04.404159 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:04.404159 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:04.404159 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:04.404159 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:04.404159 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:04.404159 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:04.404159 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:04.404159 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:04.404159 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:04.404159 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:04.404159 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:04.412811 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:04.412811 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:04.412811 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:04.412811 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:04.412811 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:04.412811 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:04.412811 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:04.412811 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:04.412811 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:04.412811 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:04.412811 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:04.412811 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:04.412811 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:04.412811 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:04.412811 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:04.412811 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:04.412811 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:04.414337 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:04.414337 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:04.414337 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:04.414337 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:04.414337 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 325/654] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:04.414337 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b29c0-c1c8-4c8a-b344-d73b6a6e2bc8 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:04.415065 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-0586d5e2-b6c6-468d-822a-60a0c5cd3b3e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 05 15:13:04.415065 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:04.415065 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:04.415065 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:04.415065 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:04.415065 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:04.415065 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:04.415065 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:04.415065 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:04.415065 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:04.415065 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:04.415065 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:04.415065 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 15:13:04.415065 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 15:13:04.415065 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 15:13:04.415065 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 15:13:04.417098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:04.417098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:04.417098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 15:13:04.417098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:04.417098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 15:13:04.417098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:04.417098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:04.417098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:04.417098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:04.417098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:04.417098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:04.417098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:04.417098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:04.417098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:04.417098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:04.417098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:04.417098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:04.417098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:04.422945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:04.422945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:04.422945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 15:13:04.422945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 15:13:04.422945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 15:13:04.422945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 15:13:04.422945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 15:13:04.422945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 15:13:04.422945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 05 15:13:04.422945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:04.422945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 330/655] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:13:04 2022] GET /identity/v3/users/glance => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:04.422945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0586d5e2-b6c6-468d-822a-60a0c5cd3b3e None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:04.512238 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:04.515386 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b29c0-c1c8-4c8a-b344-d73b6a6e2bc8 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:04.527033 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0586d5e2-b6c6-468d-822a-60a0c5cd3b3e None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:04.542139 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:04.552478 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14348ad-bf05-4e46-b202-ea55bcfcebfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:04.553369 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14348ad-bf05-4e46-b202-ea55bcfcebfd None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:04.553828 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14348ad-bf05-4e46-b202-ea55bcfcebfd None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:04.554275 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14348ad-bf05-4e46-b202-ea55bcfcebfd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:04.564398 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71714a33-6422-4e48-81fd-1f18b8eb8f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:04.565639 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71714a33-6422-4e48-81fd-1f18b8eb8f66 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:04.566014 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71714a33-6422-4e48-81fd-1f18b8eb8f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:04.566468 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71714a33-6422-4e48-81fd-1f18b8eb8f66 None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:04.567232 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71714a33-6422-4e48-81fd-1f18b8eb8f66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:04.567604 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71714a33-6422-4e48-81fd-1f18b8eb8f66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:04.569012 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14348ad-bf05-4e46-b202-ea55bcfcebfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:04.569125 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71714a33-6422-4e48-81fd-1f18b8eb8f66 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:04.570804 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14348ad-bf05-4e46-b202-ea55bcfcebfd None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:04.587354 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 331/656] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:04.587354 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71714a33-6422-4e48-81fd-1f18b8eb8f66 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:04.591036 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-e14348ad-bf05-4e46-b202-ea55bcfcebfd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:04.591036 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:04.591036 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:04.591036 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:04.591036 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:04.591036 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:04.591036 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:04.591036 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:04.591036 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:04.591036 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:04.591036 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:04.591036 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:04.591036 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:04.591036 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:04.591036 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:04.591933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:04.591933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:04.591933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:04.591933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:04.591933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:04.591933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:04.591933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:04.591933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:04.591933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:04.591933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:04.591933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:04.591933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:04.591933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:04.591933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:04.591933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:04.591933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:04.591933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:04.592873 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:04.592873 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:04.592873 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:04.592873 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:04.592873 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 326/657] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:04.593159 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14348ad-bf05-4e46-b202-ea55bcfcebfd None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:04.694753 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71714a33-6422-4e48-81fd-1f18b8eb8f66 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:04.697597 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14348ad-bf05-4e46-b202-ea55bcfcebfd None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:04.732771 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14f9c57-352e-4da1-a41a-66c6822bb85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:04.733767 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f9c57-352e-4da1-a41a-66c6822bb85f None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:04.733892 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf62329-74eb-492b-9a67-6b1dd0bbd9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:04.734371 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f9c57-352e-4da1-a41a-66c6822bb85f None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:04.734485 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf62329-74eb-492b-9a67-6b1dd0bbd9a0 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:04.734723 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf62329-74eb-492b-9a67-6b1dd0bbd9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:04.734999 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf62329-74eb-492b-9a67-6b1dd0bbd9a0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:04.735290 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f9c57-352e-4da1-a41a-66c6822bb85f None admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:04.736300 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14f9c57-352e-4da1-a41a-66c6822bb85f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:04.736904 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14f9c57-352e-4da1-a41a-66c6822bb85f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:04.738345 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14f9c57-352e-4da1-a41a-66c6822bb85f None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:04.742875 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf62329-74eb-492b-9a67-6b1dd0bbd9a0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:04.744321 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf62329-74eb-492b-9a67-6b1dd0bbd9a0 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:04.756402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-cdf62329-74eb-492b-9a67-6b1dd0bbd9a0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 15:13:04.756402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:04.756402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:04.756402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:04.756402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:04.756402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:04.756402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:04.756402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:04.756402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:04.756402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:04.756402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:04.756402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:04.756402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 15:13:04.756402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 15:13:04.756402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 15:13:04.756402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 15:13:04.757380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:04.757380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:04.757380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 15:13:04.757380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:04.757380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 15:13:04.757380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:04.757380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:04.757380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:04.757380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:04.757380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:04.757380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:04.757380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:04.757380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:04.757380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:04.757380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:04.757380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:04.757380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:04.757380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:04.758323 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:04.758323 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:04.758323 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 15:13:04.758323 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 15:13:04.758323 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 15:13:04.758323 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 15:13:04.758323 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 15:13:04.758323 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 15:13:04.758323 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 15:13:04.758323 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:04.761770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 332/658] 10.222.0.40 () {56 vars in 1137 bytes} [Fri Aug 5 15:13:04 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:04.762000 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 327/659] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 15:13:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:04.762870 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf62329-74eb-492b-9a67-6b1dd0bbd9a0 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:04.763220 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0280e33e-cdca-4313-8178-e270318ffb53 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:04.763511 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0280e33e-cdca-4313-8178-e270318ffb53 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:04.763799 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0280e33e-cdca-4313-8178-e270318ffb53 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:04.768411 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 333/660] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:04.768411 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0280e33e-cdca-4313-8178-e270318ffb53 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:04.783170 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:04.800247 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:04.869883 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf62329-74eb-492b-9a67-6b1dd0bbd9a0 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:04.873334 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0280e33e-cdca-4313-8178-e270318ffb53 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:04.907232 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f08d22b-679d-4f9d-a670-c0a560ecdd7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:04.908233 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08d22b-679d-4f9d-a670-c0a560ecdd7d None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:04.908770 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08d22b-679d-4f9d-a670-c0a560ecdd7d None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:04.909268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08d22b-679d-4f9d-a670-c0a560ecdd7d None admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:04.910227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f08d22b-679d-4f9d-a670-c0a560ecdd7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:04.910334 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b26d75-4b35-425a-b404-019ebccf595e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:04.910848 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f08d22b-679d-4f9d-a670-c0a560ecdd7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:04.910953 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b26d75-4b35-425a-b404-019ebccf595e None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:04.911183 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b26d75-4b35-425a-b404-019ebccf595e None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:04.911459 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b26d75-4b35-425a-b404-019ebccf595e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:04.912969 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f08d22b-679d-4f9d-a670-c0a560ecdd7d None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:04.926850 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 328/661] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:04.927967 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f08d22b-679d-4f9d-a670-c0a560ecdd7d None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:04.988792 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b26d75-4b35-425a-b404-019ebccf595e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:04.990244 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b26d75-4b35-425a-b404-019ebccf595e None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:05.003447 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:05.022926 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:05.033090 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f08d22b-679d-4f9d-a670-c0a560ecdd7d None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:05.069518 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987cffff-a435-4771-bc5f-ae0d49600d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:05.070123 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987cffff-a435-4771-bc5f-ae0d49600d1c None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:05.070403 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987cffff-a435-4771-bc5f-ae0d49600d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:05.070679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987cffff-a435-4771-bc5f-ae0d49600d1c None admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:05.071457 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987cffff-a435-4771-bc5f-ae0d49600d1c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:05.071761 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987cffff-a435-4771-bc5f-ae0d49600d1c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:05.073218 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987cffff-a435-4771-bc5f-ae0d49600d1c None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:05.095118 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 329/662] 10.222.0.40 () {56 vars in 1139 bytes} [Fri Aug 5 15:13:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:05.095118 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4506e-5496-4f15-88a2-68cabe42fc7f None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:05.095118 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4506e-5496-4f15-88a2-68cabe42fc7f None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:05.095118 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4506e-5496-4f15-88a2-68cabe42fc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:05.103001 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 334/663] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:13:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 15:13:05.103001 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b26d75-4b35-425a-b404-019ebccf595e None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:05.114655 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-40f4506e-5496-4f15-88a2-68cabe42fc7f None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:05.115843 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-40f4506e-5496-4f15-88a2-68cabe42fc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:05.188289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 330/664] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:13:05.201349 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f4506e-5496-4f15-88a2-68cabe42fc7f None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:05.210338 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b26d75-4b35-425a-b404-019ebccf595e None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:05.243460 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:05.248381 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161122a-cae2-44b8-bafb-ae4f1a0260f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:05.249022 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161122a-cae2-44b8-bafb-ae4f1a0260f9 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:05.249310 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161122a-cae2-44b8-bafb-ae4f1a0260f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:05.249585 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161122a-cae2-44b8-bafb-ae4f1a0260f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:05.256409 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2161122a-cae2-44b8-bafb-ae4f1a0260f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:05.257844 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2161122a-cae2-44b8-bafb-ae4f1a0260f9 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:05.267713 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:05.269111 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-2161122a-cae2-44b8-bafb-ae4f1a0260f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:05.269111 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:05.269111 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:05.269111 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:05.269111 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:05.269111 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:05.269111 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:05.269111 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:05.269111 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:05.269111 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:05.269111 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:05.269111 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:05.269111 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 15:13:05.269111 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 15:13:05.269111 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 15:13:05.269968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 15:13:05.269968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:05.269968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:05.269968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:05.269968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:05.269968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:05.269968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:05.269968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:05.269968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:05.269968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:05.269968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:05.269968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:05.269968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:05.269968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:05.269968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:05.269968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:05.269968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:05.270925 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 15:13:05.270925 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 15:13:05.270925 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 15:13:05.270925 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 15:13:05.270925 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 15:13:05.270925 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 15:13:05.270925 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:05.270925 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:05.270925 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 335/665] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:13:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:05.271398 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161122a-cae2-44b8-bafb-ae4f1a0260f9 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:05.317453 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f4506e-5496-4f15-88a2-68cabe42fc7f None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:05.353967 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd2ff1f-08e9-43c0-878c-e85771eca871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:05.354910 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2ff1f-08e9-43c0-878c-e85771eca871 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:05.355410 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2ff1f-08e9-43c0-878c-e85771eca871 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:05.355903 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2ff1f-08e9-43c0-878c-e85771eca871 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:05.386816 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161122a-cae2-44b8-bafb-ae4f1a0260f9 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:05.422523 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783bb20c-ea56-4f64-a114-c4d5712b28eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:05.423165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783bb20c-ea56-4f64-a114-c4d5712b28eb None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:05.423457 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783bb20c-ea56-4f64-a114-c4d5712b28eb None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:05.423743 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783bb20c-ea56-4f64-a114-c4d5712b28eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:05.435855 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd2ff1f-08e9-43c0-878c-e85771eca871 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:05.436000 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783bb20c-ea56-4f64-a114-c4d5712b28eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:05.437380 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783bb20c-ea56-4f64-a114-c4d5712b28eb None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:05.438072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd2ff1f-08e9-43c0-878c-e85771eca871 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:05.452405 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-783bb20c-ea56-4f64-a114-c4d5712b28eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:05.452405 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:05.452405 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:05.452405 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:05.452405 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:05.452405 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:05.452405 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:05.452405 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:05.452405 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:05.452405 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:05.452405 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:05.452405 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:05.452405 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:05.452405 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:05.452405 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:05.453299 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:05.453299 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:05.453299 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:05.453299 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:05.453299 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:05.453299 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:05.453299 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:05.453299 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:05.453299 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:05.453299 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:05.453299 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:05.453299 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:05.453299 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:05.453299 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:05.453299 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:05.453299 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:05.453299 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:05.454241 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:05.454241 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:05.454241 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:05.454241 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:05.454241 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 336/666] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:05.454993 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783bb20c-ea56-4f64-a114-c4d5712b28eb None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:05.502725 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:05.507330 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:05.551361 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 331/667] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:13:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 15:13:05.551361 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd2ff1f-08e9-43c0-878c-e85771eca871 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:05.568039 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783bb20c-ea56-4f64-a114-c4d5712b28eb None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:05.601613 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c647e-44e3-4b19-8a5a-22d0145d5135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:05.602258 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c647e-44e3-4b19-8a5a-22d0145d5135 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:05.602547 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c647e-44e3-4b19-8a5a-22d0145d5135 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:05.602832 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c647e-44e3-4b19-8a5a-22d0145d5135 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:05.613559 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c647e-44e3-4b19-8a5a-22d0145d5135 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:13:05.618919 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-711c647e-44e3-4b19-8a5a-22d0145d5135 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:13:05.618919 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:05.618919 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:05.618919 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:05.618919 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:05.618919 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:05.618919 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:05.618919 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:05.618919 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:05.618919 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:05.618919 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:05.618919 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:05.618919 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:13:05.618919 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:13:05.618919 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:13:05.618919 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:13:05.619852 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:13:05.619852 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:13:05.619852 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:13:05.619852 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:13:05.619852 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:05.619852 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:05.619852 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:05.619852 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:05.619852 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:05.619852 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:05.619852 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:05.619852 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:05.619852 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:05.619852 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:05.619852 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:05.619852 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:05.620740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:05.620740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:05.620740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:05.620740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:05.620740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:13:05.620740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:13:05.620740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:13:05.620740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:13:05.620740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:13:05.620740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:13:05.620740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:13:05.620740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:05.620740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 337/668] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 15:13:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:05.621436 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c647e-44e3-4b19-8a5a-22d0145d5135 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:05.658321 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd2ff1f-08e9-43c0-878c-e85771eca871 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:05.694395 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ca17ad-452b-4350-863a-b44edb117b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:05.695016 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca17ad-452b-4350-863a-b44edb117b20 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:05.695303 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca17ad-452b-4350-863a-b44edb117b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:05.695578 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca17ad-452b-4350-863a-b44edb117b20 None admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:05.696369 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ca17ad-452b-4350-863a-b44edb117b20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:05.696683 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ca17ad-452b-4350-863a-b44edb117b20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:05.698104 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ca17ad-452b-4350-863a-b44edb117b20 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:05.710244 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 332/669] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 15:13:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:05.713335 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ca17ad-452b-4350-863a-b44edb117b20 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:05.730540 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c647e-44e3-4b19-8a5a-22d0145d5135 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:05.733853 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:05.738240 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:05.767801 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e4718-4085-408a-90c1-3a05302c0b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:05.768450 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e4718-4085-408a-90c1-3a05302c0b52 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:05.768733 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e4718-4085-408a-90c1-3a05302c0b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:05.769016 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e4718-4085-408a-90c1-3a05302c0b52 None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:05.770051 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774e4718-4085-408a-90c1-3a05302c0b52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:05.770361 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774e4718-4085-408a-90c1-3a05302c0b52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:05.771729 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774e4718-4085-408a-90c1-3a05302c0b52 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:05.784053 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 338/670] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:05.784905 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e4718-4085-408a-90c1-3a05302c0b52 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:05.827159 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ca17ad-452b-4350-863a-b44edb117b20 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:05.864085 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8716c360-3166-4108-95fe-1520118edc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:05.864905 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8716c360-3166-4108-95fe-1520118edc84 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:05.865198 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8716c360-3166-4108-95fe-1520118edc84 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:05.865475 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8716c360-3166-4108-95fe-1520118edc84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:05.878291 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8716c360-3166-4108-95fe-1520118edc84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:05.879745 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8716c360-3166-4108-95fe-1520118edc84 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:05.889590 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e4718-4085-408a-90c1-3a05302c0b52 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:05.895511 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-8716c360-3166-4108-95fe-1520118edc84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:05.895511 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:05.895511 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:05.895511 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:05.895511 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:05.895511 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:05.895511 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:05.895511 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:05.895511 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:05.895511 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:05.895511 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:05.895511 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:05.895511 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:05.895511 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:05.895511 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:05.896464 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:05.896464 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:05.896464 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:05.896464 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:05.896464 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:05.896464 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:05.896464 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:05.896464 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:05.896464 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:05.896464 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:05.896464 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:05.896464 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:05.896464 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:05.896464 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:05.896464 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:05.896464 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:05.896464 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:05.897338 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:05.897338 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:05.897338 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:05.897338 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:05.897338 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 333/671] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:05.897626 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8716c360-3166-4108-95fe-1520118edc84 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:05.930129 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e6a96-2614-4ad3-9c83-6dd1e7a78215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:05.930754 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e6a96-2614-4ad3-9c83-6dd1e7a78215 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:05.931270 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e6a96-2614-4ad3-9c83-6dd1e7a78215 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:05.931383 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e6a96-2614-4ad3-9c83-6dd1e7a78215 None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:05.932015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e6a96-2614-4ad3-9c83-6dd1e7a78215 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:05.932366 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e6a96-2614-4ad3-9c83-6dd1e7a78215 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:05.933745 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e6a96-2614-4ad3-9c83-6dd1e7a78215 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:05.942497 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 339/672] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:13:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:05.943131 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e6a96-2614-4ad3-9c83-6dd1e7a78215 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:05.971436 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:05.982862 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:06.015597 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8716c360-3166-4108-95fe-1520118edc84 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:06.049796 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e6a96-2614-4ad3-9c83-6dd1e7a78215 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:06.055639 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9660a4-eeae-42ea-bc66-7e2be6da5c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:06.056497 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9660a4-eeae-42ea-bc66-7e2be6da5c98 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:06.056979 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9660a4-eeae-42ea-bc66-7e2be6da5c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:06.057399 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9660a4-eeae-42ea-bc66-7e2be6da5c98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:06.058326 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9660a4-eeae-42ea-bc66-7e2be6da5c98 None admin] RBAC: Adding query filter params (role.id=266a023f66e043f99efe2941fe58d481, scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83, user.id=e974f8d9a06d489884a3b00b95dd88e4) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:06.058901 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9660a4-eeae-42ea-bc66-7e2be6da5c98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:06.060626 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9660a4-eeae-42ea-bc66-7e2be6da5c98 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:06.067001 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 334/673] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:13:05 2022] GET /identity/v3/role_assignments?user.id=e974f8d9a06d489884a3b00b95dd88e4&scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83&role.id=266a023f66e043f99efe2941fe58d481 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:06.088180 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9660a4-eeae-42ea-bc66-7e2be6da5c98 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:06.094281 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbef7e03-6b87-4995-a509-1645fac268c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:06.095054 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef7e03-6b87-4995-a509-1645fac268c9 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:06.095417 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef7e03-6b87-4995-a509-1645fac268c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:06.096860 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef7e03-6b87-4995-a509-1645fac268c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:06.104720 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbef7e03-6b87-4995-a509-1645fac268c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:06.106654 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbef7e03-6b87-4995-a509-1645fac268c9 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:06.120613 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-cbef7e03-6b87-4995-a509-1645fac268c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:06.120613 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:06.120613 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:06.120613 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:06.120613 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:06.120613 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:06.120613 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:06.120613 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:06.120613 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:06.120613 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:06.120613 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:06.120613 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:06.120613 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 15:13:06.120613 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 15:13:06.120613 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 15:13:06.121787 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 15:13:06.121787 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:06.121787 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:06.121787 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:06.121787 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:06.121787 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:06.121787 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:06.121787 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:06.121787 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:06.121787 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:06.121787 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:06.121787 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:06.121787 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:06.121787 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:06.121787 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:06.121787 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:06.121787 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:06.122918 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 15:13:06.122918 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 15:13:06.122918 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 15:13:06.122918 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 15:13:06.122918 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 15:13:06.122918 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 15:13:06.122918 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:06.122918 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:06.126825 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 340/674] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:13:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:06.127704 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbef7e03-6b87-4995-a509-1645fac268c9 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:06.243717 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:06.249129 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9660a4-eeae-42ea-bc66-7e2be6da5c98 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:06.268003 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbef7e03-6b87-4995-a509-1645fac268c9 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:06.295068 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3eeb8e-f7ed-43a8-b4b7-a04fdbad51c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:06.295693 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3eeb8e-f7ed-43a8-b4b7-a04fdbad51c1 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:06.295980 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3eeb8e-f7ed-43a8-b4b7-a04fdbad51c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:06.296301 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3eeb8e-f7ed-43a8-b4b7-a04fdbad51c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:06.297007 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3eeb8e-f7ed-43a8-b4b7-a04fdbad51c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:06.297318 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3eeb8e-f7ed-43a8-b4b7-a04fdbad51c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:06.298674 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3eeb8e-f7ed-43a8-b4b7-a04fdbad51c1 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:06.303396 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:06.311990 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42356f9-fa5b-4693-a825-64f661cf975f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:06.312884 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42356f9-fa5b-4693-a825-64f661cf975f None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:06.315720 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42356f9-fa5b-4693-a825-64f661cf975f None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:06.315720 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42356f9-fa5b-4693-a825-64f661cf975f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:06.315908 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 335/675] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:06.315908 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3eeb8e-f7ed-43a8-b4b7-a04fdbad51c1 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:06.327442 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42356f9-fa5b-4693-a825-64f661cf975f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:06.329061 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42356f9-fa5b-4693-a825-64f661cf975f None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:06.344691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-e42356f9-fa5b-4693-a825-64f661cf975f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:06.344691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:06.344691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:06.344691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:06.344691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:06.344691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:06.344691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:06.344691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:06.344691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:06.344691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:06.344691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:06.344691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:06.344691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:06.344691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:06.344691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:06.345602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:06.345602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:06.345602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:06.345602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:06.345602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:06.345602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:06.345602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:06.345602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:06.345602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:06.345602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:06.345602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:06.345602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:06.345602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:06.345602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:06.345602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:06.345602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:06.345602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:06.347081 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:06.347081 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:06.347081 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:06.347081 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:06.347081 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 341/676] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:06.348129 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a07481-8695-41b2-876b-174c80ce0553 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:06.348235 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a07481-8695-41b2-876b-174c80ce0553 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:06.348478 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a07481-8695-41b2-876b-174c80ce0553 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:06.352235 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 342/677] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:06.352235 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a07481-8695-41b2-876b-174c80ce0553 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:06.430632 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3eeb8e-f7ed-43a8-b4b7-a04fdbad51c1 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:06.460021 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a07481-8695-41b2-876b-174c80ce0553 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:06.467874 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f41c23-e13e-4d00-83dd-ac984faa0c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:06.468524 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f41c23-e13e-4d00-83dd-ac984faa0c30 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:06.468808 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f41c23-e13e-4d00-83dd-ac984faa0c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:06.469085 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f41c23-e13e-4d00-83dd-ac984faa0c30 None admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:06.469845 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f41c23-e13e-4d00-83dd-ac984faa0c30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:06.470156 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f41c23-e13e-4d00-83dd-ac984faa0c30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:06.472177 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f41c23-e13e-4d00-83dd-ac984faa0c30 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:06.489445 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 336/678] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 15:13:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:06.489445 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f41c23-e13e-4d00-83dd-ac984faa0c30 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:06.503516 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15ad20-737f-441e-88af-83e4f1a26eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:06.504329 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15ad20-737f-441e-88af-83e4f1a26eac None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:06.504620 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15ad20-737f-441e-88af-83e4f1a26eac None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:06.504900 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15ad20-737f-441e-88af-83e4f1a26eac None admin] PATH_INFO: `/v3/projects/service` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:06.511952 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c15ad20-737f-441e-88af-83e4f1a26eac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:06.513442 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c15ad20-737f-441e-88af-83e4f1a26eac None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:06.515325 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:06.526002 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-3c15ad20-737f-441e-88af-83e4f1a26eac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:06.526002 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:06.526002 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:06.526002 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:06.526002 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:06.526002 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:06.526002 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:06.526002 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:06.526002 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:06.526002 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:06.526002 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:06.526002 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:06.526002 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 15:13:06.526002 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 15:13:06.526002 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 15:13:06.526906 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 15:13:06.526906 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:06.526906 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:06.526906 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:06.526906 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:06.526906 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:06.526906 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:06.526906 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:06.526906 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:06.526906 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:06.526906 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:06.526906 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:06.526906 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:06.526906 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:06.526906 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:06.526906 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:06.526906 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:06.528448 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 15:13:06.528448 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 15:13:06.528448 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 15:13:06.528448 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 15:13:06.528448 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 15:13:06.528448 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 15:13:06.528448 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:06.528448 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:06.528448 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 343/679] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:13:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:06.528953 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379acc0e-9bbf-4764-8c7f-4929daa5bbcc None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:06.528953 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379acc0e-9bbf-4764-8c7f-4929daa5bbcc None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:06.529193 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379acc0e-9bbf-4764-8c7f-4929daa5bbcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:06.547549 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-379acc0e-9bbf-4764-8c7f-4929daa5bbcc None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:06.548774 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-379acc0e-9bbf-4764-8c7f-4929daa5bbcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:06.557339 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:06.607401 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f41c23-e13e-4d00-83dd-ac984faa0c30 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:06.628246 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 344/680] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:13:06.628246 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379acc0e-9bbf-4764-8c7f-4929daa5bbcc None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:06.649703 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7debc4-4444-410c-9744-e0564a1aa609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:06.650637 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7debc4-4444-410c-9744-e0564a1aa609 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:06.651102 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7debc4-4444-410c-9744-e0564a1aa609 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:06.651670 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7debc4-4444-410c-9744-e0564a1aa609 None admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:06.652754 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7debc4-4444-410c-9744-e0564a1aa609 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:06.653276 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7debc4-4444-410c-9744-e0564a1aa609 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:06.654875 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7debc4-4444-410c-9744-e0564a1aa609 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:06.668140 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 337/681] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:06.668830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7debc4-4444-410c-9744-e0564a1aa609 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:06.746790 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379acc0e-9bbf-4764-8c7f-4929daa5bbcc None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:06.774409 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:06.776514 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7debc4-4444-410c-9744-e0564a1aa609 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:06.783058 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511ac24-6199-4d43-b244-d7fea03fd73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:06.784090 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511ac24-6199-4d43-b244-d7fea03fd73c None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:06.784394 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511ac24-6199-4d43-b244-d7fea03fd73c None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:06.785620 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511ac24-6199-4d43-b244-d7fea03fd73c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:06.785620 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5511ac24-6199-4d43-b244-d7fea03fd73c None admin] RBAC: Adding query filter params (role.id=266a023f66e043f99efe2941fe58d481, scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83, user.id=c386fa0483cf42efaddbc2395f6d1fce) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:06.786102 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5511ac24-6199-4d43-b244-d7fea03fd73c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:06.787865 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5511ac24-6199-4d43-b244-d7fea03fd73c None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:06.794262 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:06.802762 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 345/682] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:13:06 2022] GET /identity/v3/role_assignments?user.id=c386fa0483cf42efaddbc2395f6d1fce&scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83&role.id=266a023f66e043f99efe2941fe58d481 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:06.802762 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511ac24-6199-4d43-b244-d7fea03fd73c None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:06.834735 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa03c7-a8df-48c0-9b97-34eac8a2dc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:06.836036 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa03c7-a8df-48c0-9b97-34eac8a2dc0b None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:06.836673 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa03c7-a8df-48c0-9b97-34eac8a2dc0b None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:06.836956 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa03c7-a8df-48c0-9b97-34eac8a2dc0b None admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:06.837750 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fa03c7-a8df-48c0-9b97-34eac8a2dc0b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:06.838095 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fa03c7-a8df-48c0-9b97-34eac8a2dc0b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:06.841226 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fa03c7-a8df-48c0-9b97-34eac8a2dc0b None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:06.858999 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 338/683] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 15:13:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:06.859923 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa03c7-a8df-48c0-9b97-34eac8a2dc0b None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:06.924162 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511ac24-6199-4d43-b244-d7fea03fd73c None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:06.974309 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950b56a-2150-4914-bc87-8b60463153cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:06.975127 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950b56a-2150-4914-bc87-8b60463153cd None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:06.975821 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950b56a-2150-4914-bc87-8b60463153cd None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:06.976416 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950b56a-2150-4914-bc87-8b60463153cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:06.986449 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa03c7-a8df-48c0-9b97-34eac8a2dc0b None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:06.989533 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3950b56a-2150-4914-bc87-8b60463153cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:13:06.995007 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-3950b56a-2150-4914-bc87-8b60463153cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:13:06.995007 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:06.995007 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:06.995007 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:06.995007 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:06.995007 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:06.995007 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:06.995007 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:06.995007 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:06.995007 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:06.995007 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:06.995007 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:06.995007 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:13:06.995007 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:13:06.995007 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:13:06.995007 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:13:06.995933 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:13:06.995933 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:13:06.995933 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:13:06.995933 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:13:06.995933 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:06.995933 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:06.995933 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:06.995933 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:06.995933 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:06.995933 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:06.995933 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:06.995933 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:06.995933 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:06.995933 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:06.995933 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:06.995933 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:06.996866 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:06.996866 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:06.996866 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:06.996866 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:06.996866 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:13:06.996866 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:13:06.996866 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:13:06.996866 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:13:06.996866 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:13:06.996866 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:13:06.996866 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:13:06.996866 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:06.996866 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 346/684] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 15:13:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:06.997548 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950b56a-2150-4914-bc87-8b60463153cd None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:07.028728 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07697edc-fdd1-4e4a-aebb-e7bf609eaeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:07.029367 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07697edc-fdd1-4e4a-aebb-e7bf609eaeb0 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:07.029650 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07697edc-fdd1-4e4a-aebb-e7bf609eaeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:07.029943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07697edc-fdd1-4e4a-aebb-e7bf609eaeb0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:07.037959 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07697edc-fdd1-4e4a-aebb-e7bf609eaeb0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:07.039379 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07697edc-fdd1-4e4a-aebb-e7bf609eaeb0 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:07.053437 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-07697edc-fdd1-4e4a-aebb-e7bf609eaeb0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 15:13:07.053437 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:07.053437 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:07.053437 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:07.053437 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:07.053437 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:07.053437 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:07.053437 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:07.053437 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:07.053437 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:07.053437 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:07.053437 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:07.053437 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 15:13:07.053437 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 15:13:07.053437 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 15:13:07.053437 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 15:13:07.054390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:07.054390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:07.054390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 15:13:07.054390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:07.054390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 15:13:07.054390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:07.054390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:07.054390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:07.054390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:07.054390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:07.054390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:07.054390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:07.054390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:07.054390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:07.054390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:07.054390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:07.054390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:07.054390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:07.055331 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:07.055331 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:07.055331 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 15:13:07.055331 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 15:13:07.055331 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 15:13:07.055331 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 15:13:07.055331 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 15:13:07.055331 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 15:13:07.055331 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 15:13:07.055331 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:07.055331 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 339/685] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Aug 5 15:13:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:07.055331 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07697edc-fdd1-4e4a-aebb-e7bf609eaeb0 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:07.066305 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:07.109453 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:07.122221 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950b56a-2150-4914-bc87-8b60463153cd None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:07.163063 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3a69c-6046-42fe-a928-bb2fc2ba49a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:07.167360 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3a69c-6046-42fe-a928-bb2fc2ba49a3 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:07.169962 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3a69c-6046-42fe-a928-bb2fc2ba49a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:07.170231 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3a69c-6046-42fe-a928-bb2fc2ba49a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:07.171079 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d3a69c-6046-42fe-a928-bb2fc2ba49a3 None admin] RBAC: Adding query filter params (role.id=266a023f66e043f99efe2941fe58d481, scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83, user.id=d13a1638b2ca497fb02c71615559d1ea) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:07.171404 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d3a69c-6046-42fe-a928-bb2fc2ba49a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:07.175658 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d3a69c-6046-42fe-a928-bb2fc2ba49a3 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:07.182293 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 347/686] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:13:06 2022] GET /identity/v3/role_assignments?user.id=d13a1638b2ca497fb02c71615559d1ea&scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83&role.id=266a023f66e043f99efe2941fe58d481 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:07.183108 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3a69c-6046-42fe-a928-bb2fc2ba49a3 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:07.200231 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07697edc-fdd1-4e4a-aebb-e7bf609eaeb0 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:07.247798 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f1bb2-5bd8-4183-8856-192f1b41032f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:07.249064 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f1bb2-5bd8-4183-8856-192f1b41032f None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:07.251090 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f1bb2-5bd8-4183-8856-192f1b41032f None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:07.251847 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f1bb2-5bd8-4183-8856-192f1b41032f None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:07.253407 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446f1bb2-5bd8-4183-8856-192f1b41032f None admin] RBAC: Adding query filter params (name=service) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:07.254220 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446f1bb2-5bd8-4183-8856-192f1b41032f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:07.256408 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446f1bb2-5bd8-4183-8856-192f1b41032f None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:07.268201 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 340/687] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 15:13:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:07.275516 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f1bb2-5bd8-4183-8856-192f1b41032f None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:07.396791 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:07.402018 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f1bb2-5bd8-4183-8856-192f1b41032f None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:07.427449 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3a69c-6046-42fe-a928-bb2fc2ba49a3 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:07.441708 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63347781-5301-4931-95e5-0010735df1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:07.442360 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63347781-5301-4931-95e5-0010735df1ce None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:07.442646 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63347781-5301-4931-95e5-0010735df1ce None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:07.442925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63347781-5301-4931-95e5-0010735df1ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:07.451685 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:07.456537 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63347781-5301-4931-95e5-0010735df1ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:07.459034 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63347781-5301-4931-95e5-0010735df1ce None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:07.475401 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92532a8f-64bc-47bc-b79b-42631ab4d90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:07.476391 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92532a8f-64bc-47bc-b79b-42631ab4d90a None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:07.476922 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92532a8f-64bc-47bc-b79b-42631ab4d90a None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:07.477401 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92532a8f-64bc-47bc-b79b-42631ab4d90a None admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:07.477697 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-63347781-5301-4931-95e5-0010735df1ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:07.477697 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:07.477697 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:07.477697 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:07.477697 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:07.477697 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:07.477697 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:07.477697 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:07.477697 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:07.477697 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:07.477697 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:07.477697 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:07.477697 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:07.477697 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:07.477697 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:07.478544 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:07.478544 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:07.478544 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:07.478544 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:07.478544 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:07.478544 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:07.478544 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:07.478544 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:07.478544 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:07.478544 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:07.478544 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:07.478544 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:07.478544 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:07.478544 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:07.478544 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:07.478544 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:07.478544 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:07.479436 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:07.479436 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:07.479436 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:07.479436 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:07.480558 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 341/688] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:07.481014 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92532a8f-64bc-47bc-b79b-42631ab4d90a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:07.481340 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92532a8f-64bc-47bc-b79b-42631ab4d90a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:07.482773 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92532a8f-64bc-47bc-b79b-42631ab4d90a None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:07.490124 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63347781-5301-4931-95e5-0010735df1ce None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:07.517002 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 348/689] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Aug 5 15:13:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:07.523828 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92532a8f-64bc-47bc-b79b-42631ab4d90a None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:07.627333 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63347781-5301-4931-95e5-0010735df1ce None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:07.652867 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92532a8f-64bc-47bc-b79b-42631ab4d90a None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:07.673608 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0ea7fe-4b0c-48b3-a7af-9da573f5cece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:07.675449 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ea7fe-4b0c-48b3-a7af-9da573f5cece None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:07.675919 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ea7fe-4b0c-48b3-a7af-9da573f5cece None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:07.676394 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ea7fe-4b0c-48b3-a7af-9da573f5cece None admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:07.677268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0ea7fe-4b0c-48b3-a7af-9da573f5cece None admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:07.677754 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0ea7fe-4b0c-48b3-a7af-9da573f5cece None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:07.679856 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0ea7fe-4b0c-48b3-a7af-9da573f5cece None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:07.696356 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda4301-a61f-4a3f-a0a5-2beb10c184cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:07.696872 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 342/690] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:07.700211 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda4301-a61f-4a3f-a0a5-2beb10c184cf None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:07.700211 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda4301-a61f-4a3f-a0a5-2beb10c184cf None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:07.700211 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda4301-a61f-4a3f-a0a5-2beb10c184cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:07.702908 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0ea7fe-4b0c-48b3-a7af-9da573f5cece None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:07.717978 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:07.783794 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:07.825194 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0ea7fe-4b0c-48b3-a7af-9da573f5cece None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:07.845175 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccda4301-a61f-4a3f-a0a5-2beb10c184cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:07.847179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccda4301-a61f-4a3f-a0a5-2beb10c184cf None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:07.869893 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf58a3d-0e6b-4dc3-a413-cee53df33d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:07.871134 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf58a3d-0e6b-4dc3-a413-cee53df33d64 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:07.871622 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf58a3d-0e6b-4dc3-a413-cee53df33d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:07.872378 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf58a3d-0e6b-4dc3-a413-cee53df33d64 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:07.882151 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf58a3d-0e6b-4dc3-a413-cee53df33d64 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:07.883859 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf58a3d-0e6b-4dc3-a413-cee53df33d64 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:07.897318 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-eaf58a3d-0e6b-4dc3-a413-cee53df33d64 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 15:13:07.897318 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:07.897318 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:07.897318 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:07.897318 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:07.897318 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:07.897318 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:07.897318 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:07.897318 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:07.897318 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:07.897318 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:07.897318 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:07.897318 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 15:13:07.897318 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 15:13:07.897318 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 15:13:07.897318 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 15:13:07.898360 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:07.898360 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:07.898360 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 15:13:07.898360 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:07.898360 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 15:13:07.898360 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:07.898360 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:07.898360 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:07.898360 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:07.898360 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:07.898360 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:07.898360 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:07.898360 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:07.898360 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:07.898360 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:07.898360 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:07.898360 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:07.898360 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:07.899301 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:07.899301 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:07.899301 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 15:13:07.899301 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 15:13:07.899301 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 15:13:07.899301 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 15:13:07.899301 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 15:13:07.899301 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 15:13:07.899301 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 15:13:07.899301 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:07.901698 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 343/691] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 15:13:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:13:07.907456 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf58a3d-0e6b-4dc3-a413-cee53df33d64 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:07.979854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:08.013487 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf58a3d-0e6b-4dc3-a413-cee53df33d64 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:08.051347 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2158cad1-a193-42cf-86e2-1c850cf6a72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:08.052302 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158cad1-a193-42cf-86e2-1c850cf6a72d None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:08.052795 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158cad1-a193-42cf-86e2-1c850cf6a72d None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:08.053248 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158cad1-a193-42cf-86e2-1c850cf6a72d None admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:08.054215 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2158cad1-a193-42cf-86e2-1c850cf6a72d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:08.054790 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2158cad1-a193-42cf-86e2-1c850cf6a72d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:08.056402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2158cad1-a193-42cf-86e2-1c850cf6a72d None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:08.075426 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 344/692] 10.222.0.40 () {56 vars in 1135 bytes} [Fri Aug 5 15:13:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:08.081243 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2158cad1-a193-42cf-86e2-1c850cf6a72d None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:08.087218 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 349/693] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:13:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 15:13:08.092566 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda4301-a61f-4a3f-a0a5-2beb10c184cf None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:08.198298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2158cad1-a193-42cf-86e2-1c850cf6a72d None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:08.249958 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:08.259706 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d0db65-dcbf-4d28-bc28-f05b1fff2ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:08.260388 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d0db65-dcbf-4d28-bc28-f05b1fff2ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:08.260676 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d0db65-dcbf-4d28-bc28-f05b1fff2ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:08.260953 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d0db65-dcbf-4d28-bc28-f05b1fff2ff6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:08.264033 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:08.293750 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda4301-a61f-4a3f-a0a5-2beb10c184cf None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:08.340370 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d0db65-dcbf-4d28-bc28-f05b1fff2ff6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:08.340814 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b0644-c420-4943-a34d-789cb10b49b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:08.341455 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b0644-c420-4943-a34d-789cb10b49b0 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:08.341733 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b0644-c420-4943-a34d-789cb10b49b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:08.341837 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d0db65-dcbf-4d28-bc28-f05b1fff2ff6 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:08.341996 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b0644-c420-4943-a34d-789cb10b49b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:08.352808 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b0644-c420-4943-a34d-789cb10b49b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:08.354250 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b0644-c420-4943-a34d-789cb10b49b0 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:08.370419 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-d01b0644-c420-4943-a34d-789cb10b49b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:08.370419 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:08.370419 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:08.370419 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:08.370419 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:08.370419 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:08.370419 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:08.370419 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:08.370419 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:08.370419 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:08.370419 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:08.370419 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:08.370419 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:08.370419 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:08.370419 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:08.371318 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:08.371318 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:08.371318 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:08.371318 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:08.371318 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:08.371318 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:08.371318 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:08.371318 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:08.371318 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:08.371318 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:08.371318 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:08.371318 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:08.371318 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:08.371318 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:08.371318 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:08.371318 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:08.371318 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:08.372272 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:08.372272 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:08.372272 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:08.372272 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:08.372272 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 350/694] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:08.373358 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0330baf6-ec35-4f85-86b5-79ea98f2a143 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:08.373626 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0330baf6-ec35-4f85-86b5-79ea98f2a143 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:08.373899 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0330baf6-ec35-4f85-86b5-79ea98f2a143 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:08.375350 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 351/695] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:08.377214 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0330baf6-ec35-4f85-86b5-79ea98f2a143 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:08.469855 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 345/696] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:13:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 15:13:08.471854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a4114-ec63-442d-bac7-814709564847 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:08.472185 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a4114-ec63-442d-bac7-814709564847 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:08.472470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a4114-ec63-442d-bac7-814709564847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:08.492960 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-dc3a4114-ec63-442d-bac7-814709564847 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:08.494239 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-dc3a4114-ec63-442d-bac7-814709564847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:08.521114 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:08.578482 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 346/697] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:13:08.579559 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3a4114-ec63-442d-bac7-814709564847 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:08.601020 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0330baf6-ec35-4f85-86b5-79ea98f2a143 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:08.626961 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:08.635927 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad85f2b-e62f-4dc5-b390-a44d7eea8e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:08.636626 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad85f2b-e62f-4dc5-b390-a44d7eea8e05 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:08.636908 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad85f2b-e62f-4dc5-b390-a44d7eea8e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:08.637189 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad85f2b-e62f-4dc5-b390-a44d7eea8e05 None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:08.637933 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad85f2b-e62f-4dc5-b390-a44d7eea8e05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:08.638243 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad85f2b-e62f-4dc5-b390-a44d7eea8e05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:08.639653 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad85f2b-e62f-4dc5-b390-a44d7eea8e05 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:08.652480 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 352/698] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:08.653294 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad85f2b-e62f-4dc5-b390-a44d7eea8e05 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:08.717056 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3a4114-ec63-442d-bac7-814709564847 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:08.755994 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198c9bd8-1ab6-42c3-84e6-01c5b82f9216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:08.756597 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198c9bd8-1ab6-42c3-84e6-01c5b82f9216 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:08.756884 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198c9bd8-1ab6-42c3-84e6-01c5b82f9216 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:08.757058 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad85f2b-e62f-4dc5-b390-a44d7eea8e05 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:08.757225 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198c9bd8-1ab6-42c3-84e6-01c5b82f9216 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:08.768851 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198c9bd8-1ab6-42c3-84e6-01c5b82f9216 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:08.770388 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198c9bd8-1ab6-42c3-84e6-01c5b82f9216 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:08.787346 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-198c9bd8-1ab6-42c3-84e6-01c5b82f9216 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:08.787346 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:08.787346 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:08.787346 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:08.787346 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:08.787346 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:08.787346 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:08.787346 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:08.787346 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:08.787346 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:08.787346 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:08.787346 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:08.787346 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:08.787346 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:08.787346 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:08.789417 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:08.789417 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:08.789417 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:08.789417 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:08.789417 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:08.789417 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:08.789417 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:08.789417 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:08.789417 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:08.789417 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:08.789417 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:08.789417 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:08.789417 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:08.789417 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:08.789417 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:08.789417 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:08.789417 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:08.791890 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:08.791890 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:08.791890 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:08.791890 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:08.791890 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 347/699] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:08.791890 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198c9bd8-1ab6-42c3-84e6-01c5b82f9216 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:08.792763 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ddfdb-19cf-4d37-9e03-38f033b1aeea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:08.797202 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ddfdb-19cf-4d37-9e03-38f033b1aeea None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:08.797202 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ddfdb-19cf-4d37-9e03-38f033b1aeea None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:08.797202 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ddfdb-19cf-4d37-9e03-38f033b1aeea None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:08.797202 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327ddfdb-19cf-4d37-9e03-38f033b1aeea None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:08.797202 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327ddfdb-19cf-4d37-9e03-38f033b1aeea None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:08.802515 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:08.808566 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-327ddfdb-19cf-4d37-9e03-38f033b1aeea None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 15:13:08.808566 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:08.808566 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 15:13:08.808566 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 15:13:08.808566 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 15:13:08.808566 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 15:13:08.808566 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 15:13:08.808566 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:08.808566 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 15:13:08.808566 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:08.808566 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 15:13:08.808566 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:08.808566 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:08.808566 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:08.808566 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:08.808566 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:08.816267 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:08.816267 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:08.816267 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:08.816267 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:08.816267 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:08.816267 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:08.816267 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:08.816267 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 15:13:08.816267 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 15:13:08.816267 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 15:13:08.816267 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 15:13:08.816267 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:08.816267 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:08.816267 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:08.816267 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:08.816267 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:08.818266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:08.818266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:08.818266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:08.818266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:08.818266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:08.818266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:08.818266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:08.818266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:08.818266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:08.818266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:08.818266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:08.818266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 15:13:08.818266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:08.818266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 15:13:08.818266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:08.818266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 353/700] 10.222.0.40 () {56 vars in 1129 bytes} [Fri Aug 5 15:13:08 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:08.820292 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ddfdb-19cf-4d37-9e03-38f033b1aeea None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:08.868868 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:08.904762 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198c9bd8-1ab6-42c3-84e6-01c5b82f9216 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:08.920938 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ddfdb-19cf-4d37-9e03-38f033b1aeea None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:08.943326 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f1376c-23b3-46e5-bef3-dfcf4d4094e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:08.943972 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1376c-23b3-46e5-bef3-dfcf4d4094e7 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:08.944289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1376c-23b3-46e5-bef3-dfcf4d4094e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:08.944591 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1376c-23b3-46e5-bef3-dfcf4d4094e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:08.952263 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f1376c-23b3-46e5-bef3-dfcf4d4094e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:08.953704 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f1376c-23b3-46e5-bef3-dfcf4d4094e7 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:08.957053 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a93a4b-e6b4-4014-9cd0-3205a0f2a259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:08.957664 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a93a4b-e6b4-4014-9cd0-3205a0f2a259 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:08.957942 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a93a4b-e6b4-4014-9cd0-3205a0f2a259 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:08.958216 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a93a4b-e6b4-4014-9cd0-3205a0f2a259 None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:08.958905 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a93a4b-e6b4-4014-9cd0-3205a0f2a259 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:08.959207 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a93a4b-e6b4-4014-9cd0-3205a0f2a259 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:08.960668 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a93a4b-e6b4-4014-9cd0-3205a0f2a259 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:08.966241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-72f1376c-23b3-46e5-bef3-dfcf4d4094e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:08.966241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:08.966241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:08.966241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:08.966241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:08.966241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:08.966241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:08.966241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:08.966241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:08.966241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:08.966241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:08.966241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:08.966241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 15:13:08.966241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 15:13:08.966241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 15:13:08.968300 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 15:13:08.968300 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:08.968300 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:08.968300 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:08.968300 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:08.968300 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:08.968300 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:08.968300 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:08.968300 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:08.968300 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:08.968300 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:08.968300 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:08.968300 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:08.968300 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:08.968300 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:08.968300 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:08.968300 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:08.970407 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 15:13:08.970407 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 15:13:08.970407 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 15:13:08.970407 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 15:13:08.970407 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 15:13:08.970407 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 15:13:08.970407 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:08.970407 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:08.976183 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 348/701] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:13:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:08.976183 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f1376c-23b3-46e5-bef3-dfcf4d4094e7 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:08.978960 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 354/702] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:08.979809 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a93a4b-e6b4-4014-9cd0-3205a0f2a259 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:09.065399 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:09.082999 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a93a4b-e6b4-4014-9cd0-3205a0f2a259 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:09.094003 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:09.102277 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f1376c-23b3-46e5-bef3-dfcf4d4094e7 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:09.118665 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c29eb-9826-4a50-9a43-e8c0c6abf336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:09.119283 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c29eb-9826-4a50-9a43-e8c0c6abf336 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:09.119565 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c29eb-9826-4a50-9a43-e8c0c6abf336 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:09.119838 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c29eb-9826-4a50-9a43-e8c0c6abf336 None admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:09.120614 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c29eb-9826-4a50-9a43-e8c0c6abf336 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:09.120928 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c29eb-9826-4a50-9a43-e8c0c6abf336 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:09.122402 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c29eb-9826-4a50-9a43-e8c0c6abf336 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:09.139662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 355/703] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 15:13:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:09.139662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c29eb-9826-4a50-9a43-e8c0c6abf336 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:09.144395 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e93b54-5efd-4862-a9cd-e175054d5156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:09.145023 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e93b54-5efd-4862-a9cd-e175054d5156 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:09.145316 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e93b54-5efd-4862-a9cd-e175054d5156 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:09.145598 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e93b54-5efd-4862-a9cd-e175054d5156 None admin] PATH_INFO: `/v3/services` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:09.146418 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e93b54-5efd-4862-a9cd-e175054d5156 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:09.146726 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e93b54-5efd-4862-a9cd-e175054d5156 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:09.148248 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e93b54-5efd-4862-a9cd-e175054d5156 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:09.161439 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 349/704] 10.222.0.40 () {56 vars in 1124 bytes} [Fri Aug 5 15:13:08 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:09.161439 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b518e5-fc79-436e-85c5-6cd3df5826af None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:09.161439 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b518e5-fc79-436e-85c5-6cd3df5826af None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:09.161439 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b518e5-fc79-436e-85c5-6cd3df5826af None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:09.161439 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 350/705] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:09.161439 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b518e5-fc79-436e-85c5-6cd3df5826af None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:09.258817 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c29eb-9826-4a50-9a43-e8c0c6abf336 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:09.301295 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5740008-e644-4ce2-bc17-4b7dc8f810fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:09.301965 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5740008-e644-4ce2-bc17-4b7dc8f810fd None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:09.302250 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5740008-e644-4ce2-bc17-4b7dc8f810fd None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:09.302524 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5740008-e644-4ce2-bc17-4b7dc8f810fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:09.310549 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5740008-e644-4ce2-bc17-4b7dc8f810fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:09.312042 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5740008-e644-4ce2-bc17-4b7dc8f810fd None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:09.325370 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-f5740008-e644-4ce2-bc17-4b7dc8f810fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:09.325370 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:09.325370 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:09.325370 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:09.325370 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:09.325370 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:09.325370 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:09.325370 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:09.325370 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:09.325370 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:09.325370 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:09.325370 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:09.325370 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 15:13:09.325370 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 15:13:09.325370 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 15:13:09.326308 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 15:13:09.326308 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:09.326308 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:09.326308 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:09.326308 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:09.326308 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:09.326308 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:09.326308 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:09.326308 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:09.326308 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:09.326308 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:09.326308 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:09.326308 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:09.326308 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:09.326308 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:09.326308 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:09.326308 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:09.327191 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 15:13:09.327191 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 15:13:09.327191 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 15:13:09.327191 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 15:13:09.327191 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 15:13:09.327191 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 15:13:09.327191 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:09.327191 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:09.331380 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 356/706] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:13:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:09.332696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b10f6-7c57-4223-b8e2-ec84813d999d None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:09.332989 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b10f6-7c57-4223-b8e2-ec84813d999d None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:09.333269 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b10f6-7c57-4223-b8e2-ec84813d999d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:09.352327 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-498b10f6-7c57-4223-b8e2-ec84813d999d None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:09.353547 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-498b10f6-7c57-4223-b8e2-ec84813d999d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:09.380449 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:09.393109 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b518e5-fc79-436e-85c5-6cd3df5826af None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:09.436094 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 357/707] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:13:09.436094 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b10f6-7c57-4223-b8e2-ec84813d999d None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:09.439175 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8df1a4-3d2a-4cf6-9b3c-873552194b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:09.440391 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8df1a4-3d2a-4cf6-9b3c-873552194b8f None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:09.440677 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8df1a4-3d2a-4cf6-9b3c-873552194b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:09.440955 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8df1a4-3d2a-4cf6-9b3c-873552194b8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:09.441753 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8df1a4-3d2a-4cf6-9b3c-873552194b8f None admin] RBAC: Adding query filter params (role.id=7c88c815db854954b516ab97cd471647, scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83, user.id=bc0497dab1f44bb7b8196d39bf01d770) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:09.442067 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8df1a4-3d2a-4cf6-9b3c-873552194b8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:09.443483 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8df1a4-3d2a-4cf6-9b3c-873552194b8f None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:09.445595 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:09.460267 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 351/708] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:13:09 2022] GET /identity/v3/role_assignments?user.id=bc0497dab1f44bb7b8196d39bf01d770&scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83&role.id=7c88c815db854954b516ab97cd471647 => generated 259 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:09.461580 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8df1a4-3d2a-4cf6-9b3c-873552194b8f None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:09.563486 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b10f6-7c57-4223-b8e2-ec84813d999d None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:09.601541 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac905afc-e3eb-4389-841d-1df4de40c236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:09.602172 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac905afc-e3eb-4389-841d-1df4de40c236 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:09.602355 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac905afc-e3eb-4389-841d-1df4de40c236 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:09.602619 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac905afc-e3eb-4389-841d-1df4de40c236 None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:09.603309 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac905afc-e3eb-4389-841d-1df4de40c236 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:09.603858 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac905afc-e3eb-4389-841d-1df4de40c236 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:09.605357 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac905afc-e3eb-4389-841d-1df4de40c236 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:09.616753 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8df1a4-3d2a-4cf6-9b3c-873552194b8f None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:09.620417 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 358/709] 10.222.0.40 () {56 vars in 1124 bytes} [Fri Aug 5 15:13:09 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:09.621273 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac905afc-e3eb-4389-841d-1df4de40c236 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:09.663907 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:09.669596 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e813826d-afdd-4334-9b3f-554543dd13e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:09.670585 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813826d-afdd-4334-9b3f-554543dd13e1 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:09.671210 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813826d-afdd-4334-9b3f-554543dd13e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:09.672207 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813826d-afdd-4334-9b3f-554543dd13e1 None admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:09.677495 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e813826d-afdd-4334-9b3f-554543dd13e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:09.677495 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e813826d-afdd-4334-9b3f-554543dd13e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:09.677495 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e813826d-afdd-4334-9b3f-554543dd13e1 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:09.693895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 352/710] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 15:13:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:09.700937 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e813826d-afdd-4334-9b3f-554543dd13e1 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:09.755579 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac905afc-e3eb-4389-841d-1df4de40c236 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:09.772990 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:09.794260 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108a37c8-6b48-452d-b2b4-ddf0b8bc114a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:09.794972 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a37c8-6b48-452d-b2b4-ddf0b8bc114a None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:09.795260 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a37c8-6b48-452d-b2b4-ddf0b8bc114a None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:09.795534 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a37c8-6b48-452d-b2b4-ddf0b8bc114a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:09.811975 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108a37c8-6b48-452d-b2b4-ddf0b8bc114a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:09.814257 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108a37c8-6b48-452d-b2b4-ddf0b8bc114a None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:09.834411 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-108a37c8-6b48-452d-b2b4-ddf0b8bc114a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:09.834411 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:09.834411 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:09.834411 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:09.834411 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:09.834411 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:09.834411 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:09.834411 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:09.834411 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:09.834411 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:09.834411 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:09.834411 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:09.834411 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:09.834411 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:09.834411 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:09.835454 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:09.835454 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:09.835454 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:09.835454 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:09.835454 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:09.835454 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:09.835454 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:09.835454 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:09.835454 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:09.835454 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:09.835454 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:09.835454 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:09.835454 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:09.835454 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:09.835454 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:09.835454 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:09.835454 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:09.836464 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:09.836464 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:09.836464 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:09.836464 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:09.836464 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 359/711] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:09.837804 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e813826d-afdd-4334-9b3f-554543dd13e1 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:09.841877 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c43fc1-3f83-4979-b7bf-353b9e851712 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:09.842163 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c43fc1-3f83-4979-b7bf-353b9e851712 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:09.842495 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c43fc1-3f83-4979-b7bf-353b9e851712 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:09.843958 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 360/712] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:09.844534 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c43fc1-3f83-4979-b7bf-353b9e851712 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:09.887435 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c76401f-d777-48ac-83f7-8d0afbb2c735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:09.888476 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76401f-d777-48ac-83f7-8d0afbb2c735 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:09.888986 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76401f-d777-48ac-83f7-8d0afbb2c735 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:09.889632 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76401f-d777-48ac-83f7-8d0afbb2c735 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:09.890647 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c76401f-d777-48ac-83f7-8d0afbb2c735 None admin] RBAC: Adding query filter params (role.id=266a023f66e043f99efe2941fe58d481, scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83, user.id=85154ce7446245738edcc8702dbf6861) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:09.891988 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c76401f-d777-48ac-83f7-8d0afbb2c735 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:09.894229 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c76401f-d777-48ac-83f7-8d0afbb2c735 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:09.902932 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 353/713] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:13:09 2022] GET /identity/v3/role_assignments?user.id=85154ce7446245738edcc8702dbf6861&scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83&role.id=266a023f66e043f99efe2941fe58d481 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:09.905360 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3595eb8-80fe-47cd-ad22-c71e9550e4fd None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:09.905927 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3595eb8-80fe-47cd-ad22-c71e9550e4fd None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:09.906673 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3595eb8-80fe-47cd-ad22-c71e9550e4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:09.932186 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-a3595eb8-80fe-47cd-ad22-c71e9550e4fd None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:09.933828 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-a3595eb8-80fe-47cd-ad22-c71e9550e4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:09.939928 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:09.970782 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c43fc1-3f83-4979-b7bf-353b9e851712 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:10.023603 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58fe7b6-81ef-4451-975f-0d73698efbb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:10.025086 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58fe7b6-81ef-4451-975f-0d73698efbb5 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:10.025749 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58fe7b6-81ef-4451-975f-0d73698efbb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:10.026244 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58fe7b6-81ef-4451-975f-0d73698efbb5 None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:10.027216 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58fe7b6-81ef-4451-975f-0d73698efbb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:10.027739 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58fe7b6-81ef-4451-975f-0d73698efbb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:10.029375 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58fe7b6-81ef-4451-975f-0d73698efbb5 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:10.036086 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 354/714] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:13:10.049287 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 361/715] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:10.054829 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3595eb8-80fe-47cd-ad22-c71e9550e4fd None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:10.057190 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58fe7b6-81ef-4451-975f-0d73698efbb5 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:10.114045 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:10.164035 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3595eb8-80fe-47cd-ad22-c71e9550e4fd None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:10.202780 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58fe7b6-81ef-4451-975f-0d73698efbb5 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:10.209657 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03861580-dd98-40d5-9a59-130fc2c67131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:10.210651 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03861580-dd98-40d5-9a59-130fc2c67131 None admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:10.211192 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03861580-dd98-40d5-9a59-130fc2c67131 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:10.212197 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03861580-dd98-40d5-9a59-130fc2c67131 None admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:10.213101 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03861580-dd98-40d5-9a59-130fc2c67131 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:10.215688 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03861580-dd98-40d5-9a59-130fc2c67131 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:10.248919 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f5def-b8f0-4c0c-b85b-5eecaedd3392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:10.250288 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f5def-b8f0-4c0c-b85b-5eecaedd3392 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:10.250812 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f5def-b8f0-4c0c-b85b-5eecaedd3392 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:10.251300 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f5def-b8f0-4c0c-b85b-5eecaedd3392 None admin] PATH_INFO: `/v3/services/network` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:10.252752 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6f5def-b8f0-4c0c-b85b-5eecaedd3392 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:10.254440 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6f5def-b8f0-4c0c-b85b-5eecaedd3392 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:10.257962 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:10.265236 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-6e6f5def-b8f0-4c0c-b85b-5eecaedd3392 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 15:13:10.265236 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:10.265236 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 15:13:10.265236 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 15:13:10.265236 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 15:13:10.265236 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 15:13:10.265236 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 15:13:10.265236 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:10.265236 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 15:13:10.265236 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:10.265236 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 15:13:10.265236 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:10.265236 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:10.265236 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:10.265236 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:10.265236 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:10.265236 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:10.266390 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:10.266390 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:10.266390 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:10.266390 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:10.266390 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:10.266390 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:10.266390 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 15:13:10.266390 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 15:13:10.266390 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 15:13:10.266390 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 15:13:10.266390 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:10.266390 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:10.266390 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:10.266390 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:10.266390 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:10.266390 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:10.267251 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:10.267251 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:10.267251 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:10.267251 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:10.267251 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:10.267251 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:10.267251 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:10.267251 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:10.267251 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:10.267251 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:10.267251 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 15:13:10.267251 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:10.267251 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 15:13:10.267251 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:10.269926 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 362/716] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:13:10 2022] GET /identity/v3/services/network => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:10.276312 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f5def-b8f0-4c0c-b85b-5eecaedd3392 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:10.297398 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 355/717] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 5 15:13:10 2022] POST /identity/v3/users => generated 254 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:13:10.400542 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f5def-b8f0-4c0c-b85b-5eecaedd3392 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:10.438164 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee380114-e92c-446e-87bf-739f41abe060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:10.438714 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee380114-e92c-446e-87bf-739f41abe060 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:10.438997 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee380114-e92c-446e-87bf-739f41abe060 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:10.439375 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee380114-e92c-446e-87bf-739f41abe060 None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:10.440089 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee380114-e92c-446e-87bf-739f41abe060 None admin] RBAC: Adding query filter params (name=network) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:10.440399 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee380114-e92c-446e-87bf-739f41abe060 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:10.441798 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee380114-e92c-446e-87bf-739f41abe060 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:10.448042 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 363/718] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 5 15:13:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:10.453651 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03861580-dd98-40d5-9a59-130fc2c67131 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:10.569569 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03861580-dd98-40d5-9a59-130fc2c67131 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:10.579402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:10.608001 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241db766-1b28-4f49-88ef-f68274ee85f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:10.608699 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241db766-1b28-4f49-88ef-f68274ee85f6 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:10.609150 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241db766-1b28-4f49-88ef-f68274ee85f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:10.609430 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241db766-1b28-4f49-88ef-f68274ee85f6 None admin] PATH_INFO: `/v3/services` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:10.610168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241db766-1b28-4f49-88ef-f68274ee85f6 None admin] RBAC: Adding query filter params (type=network) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:10.610846 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241db766-1b28-4f49-88ef-f68274ee85f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:10.613035 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241db766-1b28-4f49-88ef-f68274ee85f6 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:10.619838 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 356/719] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 5 15:13:10 2022] GET /identity/v3/services?type=network => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:11.856497 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acbc02f-8182-496e-b767-2791174c3953 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:11.857051 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acbc02f-8182-496e-b767-2791174c3953 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:11.857051 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acbc02f-8182-496e-b767-2791174c3953 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:11.858653 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 364/720] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:11.868532 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936bda3e-76eb-4232-b830-f1632759b484 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:11.868970 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936bda3e-76eb-4232-b830-f1632759b484 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:11.869142 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936bda3e-76eb-4232-b830-f1632759b484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:11.896813 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-936bda3e-76eb-4232-b830-f1632759b484 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:11.898057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-936bda3e-76eb-4232-b830-f1632759b484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:11.967082 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 357/721] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:13:11.990087 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acbc02f-8182-496e-b767-2791174c3953 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:12.070648 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:12.111527 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acbc02f-8182-496e-b767-2791174c3953 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:12.145174 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab871b12-f149-427b-b046-cb64e4efe71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:12.145803 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab871b12-f149-427b-b046-cb64e4efe71d None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:12.146085 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab871b12-f149-427b-b046-cb64e4efe71d None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:12.146360 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab871b12-f149-427b-b046-cb64e4efe71d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:12.180743 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab871b12-f149-427b-b046-cb64e4efe71d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:13:12.186394 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079cc1d-de6d-4a9c-974e-d84bb7c2b8c1 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:12.186684 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079cc1d-de6d-4a9c-974e-d84bb7c2b8c1 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:12.186959 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079cc1d-de6d-4a9c-974e-d84bb7c2b8c1 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:12.188332 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 358/722] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 15:13:12.189730 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-ab871b12-f149-427b-b046-cb64e4efe71d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:13:12.189730 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:12.189730 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:12.189730 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:12.189730 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:12.189730 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:12.189730 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:12.189730 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:12.189730 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:12.189730 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:12.189730 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:12.189730 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:12.189730 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:13:12.189730 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:13:12.189730 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:13:12.189730 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:13:12.190673 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:13:12.190673 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:13:12.190673 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:13:12.190673 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:13:12.190673 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:12.190673 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:12.190673 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:12.190673 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:12.190673 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:12.190673 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:12.190673 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:12.190673 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:12.190673 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:12.190673 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:12.190673 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:12.190673 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:12.192204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:12.192204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:12.192204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:12.192204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:12.192204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:13:12.192204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:13:12.192204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:13:12.192204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:13:12.192204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:13:12.192204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:13:12.192204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:13:12.192204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:12.192204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 365/723] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 15:13:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:12.193889 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7416f-5523-42a7-8295-de1f827a58cb None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:12.193889 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7416f-5523-42a7-8295-de1f827a58cb None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:12.193889 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7416f-5523-42a7-8295-de1f827a58cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:12.202659 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab871b12-f149-427b-b046-cb64e4efe71d None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:12.224150 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-75c7416f-5523-42a7-8295-de1f827a58cb None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:12.225344 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-75c7416f-5523-42a7-8295-de1f827a58cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:12.318075 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 359/724] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:13:12.342861 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7416f-5523-42a7-8295-de1f827a58cb None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:12.374914 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:12.377046 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab871b12-f149-427b-b046-cb64e4efe71d None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:12.401346 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:12.413786 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a864f9e-f0cc-4a2a-a3e2-a2f3fcb4d859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:12.414908 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a864f9e-f0cc-4a2a-a3e2-a2f3fcb4d859 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:12.415410 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a864f9e-f0cc-4a2a-a3e2-a2f3fcb4d859 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:12.415853 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a864f9e-f0cc-4a2a-a3e2-a2f3fcb4d859 None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:12.416834 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a864f9e-f0cc-4a2a-a3e2-a2f3fcb4d859 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:12.417305 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a864f9e-f0cc-4a2a-a3e2-a2f3fcb4d859 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:12.418960 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a864f9e-f0cc-4a2a-a3e2-a2f3fcb4d859 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:12.429627 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 366/725] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:13:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:12.435017 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a864f9e-f0cc-4a2a-a3e2-a2f3fcb4d859 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:12.486171 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7416f-5523-42a7-8295-de1f827a58cb None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:12.524110 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5415cbcf-9879-4719-ac28-0552465503da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:12.525017 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415cbcf-9879-4719-ac28-0552465503da None admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:12.525477 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415cbcf-9879-4719-ac28-0552465503da None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:12.525950 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415cbcf-9879-4719-ac28-0552465503da None admin] PATH_INFO: `/v3/services` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:12.526899 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5415cbcf-9879-4719-ac28-0552465503da None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:12.528551 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5415cbcf-9879-4719-ac28-0552465503da None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:12.603278 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a864f9e-f0cc-4a2a-a3e2-a2f3fcb4d859 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:12.632085 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 360/726] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 5 15:13:12 2022] POST /identity/v3/services => generated 251 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:13:12.656303 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423f3f58-2b06-4491-b3fd-04047dfebdde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:12.657313 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f3f58-2b06-4491-b3fd-04047dfebdde None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:12.658336 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f3f58-2b06-4491-b3fd-04047dfebdde None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:12.658802 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f3f58-2b06-4491-b3fd-04047dfebdde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:12.671027 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423f3f58-2b06-4491-b3fd-04047dfebdde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:12.672838 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423f3f58-2b06-4491-b3fd-04047dfebdde None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:12.691489 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-423f3f58-2b06-4491-b3fd-04047dfebdde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:12.691489 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:12.691489 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:12.691489 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:12.691489 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:12.691489 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:12.691489 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:12.691489 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:12.691489 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:12.691489 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:12.691489 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:12.691489 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:12.691489 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:12.691489 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:12.691489 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:12.692487 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:12.692487 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:12.692487 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:12.692487 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:12.692487 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:12.692487 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:12.692487 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:12.692487 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:12.692487 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:12.692487 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:12.692487 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:12.692487 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:12.692487 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:12.692487 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:12.692487 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:12.692487 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:12.692487 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:12.693406 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:12.693406 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:12.693406 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:12.693406 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:12.695442 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 367/727] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:12.705946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5415cbcf-9879-4719-ac28-0552465503da None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:12.755988 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4324b-149b-4cf5-af51-c3e1d9d78a6b None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:12.756682 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4324b-149b-4cf5-af51-c3e1d9d78a6b None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:12.757195 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4324b-149b-4cf5-af51-c3e1d9d78a6b None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:12.759447 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 368/728] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 15:13:12.770293 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99f78c-3b44-445b-a47a-844dbedcc326 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:12.770918 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99f78c-3b44-445b-a47a-844dbedcc326 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:12.771409 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99f78c-3b44-445b-a47a-844dbedcc326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:12.799428 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-8c99f78c-3b44-445b-a47a-844dbedcc326 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:12.809508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-8c99f78c-3b44-445b-a47a-844dbedcc326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:12.849958 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:12.851303 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:12.875204 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5415cbcf-9879-4719-ac28-0552465503da None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:12.912173 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 369/729] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:12 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 15:13:12.926896 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5643ef4f-145c-4964-983e-ef1d25a47622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:12.927687 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643ef4f-145c-4964-983e-ef1d25a47622 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:12.927983 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643ef4f-145c-4964-983e-ef1d25a47622 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:12.928296 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643ef4f-145c-4964-983e-ef1d25a47622 None admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:12.929009 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5643ef4f-145c-4964-983e-ef1d25a47622 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:12.929323 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5643ef4f-145c-4964-983e-ef1d25a47622 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:12.930741 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5643ef4f-145c-4964-983e-ef1d25a47622 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:12.934637 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99f78c-3b44-445b-a47a-844dbedcc326 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:12.952382 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 361/730] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:12.958316 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5643ef4f-145c-4964-983e-ef1d25a47622 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:13.058454 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99f78c-3b44-445b-a47a-844dbedcc326 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:13.082662 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5643ef4f-145c-4964-983e-ef1d25a47622 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:13.098020 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070758e2-2c43-491a-bec2-8fa79adc3f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:13.098995 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070758e2-2c43-491a-bec2-8fa79adc3f62 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:13.099490 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070758e2-2c43-491a-bec2-8fa79adc3f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:13.100140 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070758e2-2c43-491a-bec2-8fa79adc3f62 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:13.112419 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070758e2-2c43-491a-bec2-8fa79adc3f62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:13:13.118368 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-070758e2-2c43-491a-bec2-8fa79adc3f62 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:13:13.118368 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:13.118368 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:13.118368 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:13.118368 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:13.118368 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:13.118368 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:13.118368 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:13.118368 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:13.118368 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:13.118368 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:13.118368 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:13.118368 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:13:13.118368 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:13:13.118368 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:13:13.118368 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:13:13.119311 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:13:13.119311 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:13:13.119311 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:13:13.119311 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:13:13.119311 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:13.119311 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:13.119311 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:13.119311 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:13.119311 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:13.119311 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:13.119311 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:13.119311 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:13.119311 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:13.119311 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:13.119311 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:13.119311 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:13.120186 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:13.120186 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:13.120186 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:13.120186 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:13.120186 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:13:13.120186 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:13:13.120186 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:13:13.120186 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:13:13.120186 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:13:13.120186 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:13:13.120186 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:13:13.120186 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:13.120849 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3161c6-35f9-494e-96fc-21eb1137493a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:13.120849 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3161c6-35f9-494e-96fc-21eb1137493a None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:13.120849 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3161c6-35f9-494e-96fc-21eb1137493a None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:13.120849 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3161c6-35f9-494e-96fc-21eb1137493a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:13.128189 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 370/731] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 15:13:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:13.130423 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3161c6-35f9-494e-96fc-21eb1137493a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:13.130767 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070758e2-2c43-491a-bec2-8fa79adc3f62 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:13.132099 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3161c6-35f9-494e-96fc-21eb1137493a None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:13.144787 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-9a3161c6-35f9-494e-96fc-21eb1137493a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 15:13:13.144787 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:13.144787 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:13.144787 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:13.144787 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:13.144787 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:13.144787 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:13.144787 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:13.144787 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:13.144787 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:13.144787 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:13.144787 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:13.144787 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 15:13:13.144787 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 15:13:13.144787 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 15:13:13.144787 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 15:13:13.145817 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:13.145817 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:13.145817 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 15:13:13.145817 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:13.145817 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 15:13:13.145817 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:13.145817 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:13.145817 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:13.145817 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:13.145817 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:13.145817 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:13.145817 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:13.145817 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:13.145817 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:13.145817 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:13.145817 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:13.145817 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:13.145817 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:13.147407 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:13.147407 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:13.147407 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 15:13:13.147407 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 15:13:13.147407 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 15:13:13.147407 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 15:13:13.147407 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 15:13:13.147407 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 15:13:13.147407 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 15:13:13.147407 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:13.147407 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 362/732] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Aug 5 15:13:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:13.151867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3161c6-35f9-494e-96fc-21eb1137493a None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:13.164022 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:13.167479 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:13.248229 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070758e2-2c43-491a-bec2-8fa79adc3f62 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:13.260313 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3161c6-35f9-494e-96fc-21eb1137493a None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:13.284092 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca5531a-47ae-4ea2-871d-afb5cbf829c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:13.284719 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca5531a-47ae-4ea2-871d-afb5cbf829c8 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:13.285005 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca5531a-47ae-4ea2-871d-afb5cbf829c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:13.285282 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca5531a-47ae-4ea2-871d-afb5cbf829c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:13.286024 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca5531a-47ae-4ea2-871d-afb5cbf829c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:13.286333 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca5531a-47ae-4ea2-871d-afb5cbf829c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:13.287860 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca5531a-47ae-4ea2-871d-afb5cbf829c8 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:13.295703 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bf4e13-5731-492a-b81c-5f439daaca88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:13.296347 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf4e13-5731-492a-b81c-5f439daaca88 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:13.296634 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf4e13-5731-492a-b81c-5f439daaca88 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:13.296911 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf4e13-5731-492a-b81c-5f439daaca88 None admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:13.297718 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bf4e13-5731-492a-b81c-5f439daaca88 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:13.298031 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bf4e13-5731-492a-b81c-5f439daaca88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:13.299988 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bf4e13-5731-492a-b81c-5f439daaca88 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:13.301641 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 371/733] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:13:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:13.303699 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f807d-7fa7-4493-a527-f73961560766 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:13.303699 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f807d-7fa7-4493-a527-f73961560766 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:13.303699 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f807d-7fa7-4493-a527-f73961560766 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:13.308970 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 372/734] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:13.308970 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9fc85-00e2-40a2-a964-5e975c29629c None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:13.308970 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9fc85-00e2-40a2-a964-5e975c29629c None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:13.308970 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9fc85-00e2-40a2-a964-5e975c29629c None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:13.308970 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 373/735] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:13.308970 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d9fc85-00e2-40a2-a964-5e975c29629c None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:13.328887 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 363/736] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Aug 5 15:13:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:13.328887 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a6a49-f058-4522-9b3d-e918f0859754 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:13.328887 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a6a49-f058-4522-9b3d-e918f0859754 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:13.328887 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a6a49-f058-4522-9b3d-e918f0859754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:13.343925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-c12a6a49-f058-4522-9b3d-e918f0859754 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:13.345175 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-c12a6a49-f058-4522-9b3d-e918f0859754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:13.415857 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:13.416463 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:13.429928 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 364/737] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:13 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:13:13.429928 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc668a99-52b4-4146-b390-b5c8f9f0791a None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:13.429928 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc668a99-52b4-4146-b390-b5c8f9f0791a None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:13.429928 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc668a99-52b4-4146-b390-b5c8f9f0791a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:13.442848 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d9fc85-00e2-40a2-a964-5e975c29629c None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:13.448989 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-cc668a99-52b4-4146-b390-b5c8f9f0791a None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:13.450407 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-cc668a99-52b4-4146-b390-b5c8f9f0791a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:13.478083 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3d9d67-2ae2-4e51-b3b3-65c1ae9c83d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:13.478721 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3d9d67-2ae2-4e51-b3b3-65c1ae9c83d7 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:13.479008 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3d9d67-2ae2-4e51-b3b3-65c1ae9c83d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:13.479291 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3d9d67-2ae2-4e51-b3b3-65c1ae9c83d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:13.490436 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3d9d67-2ae2-4e51-b3b3-65c1ae9c83d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:13.491893 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3d9d67-2ae2-4e51-b3b3-65c1ae9c83d7 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:13.506838 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-3c3d9d67-2ae2-4e51-b3b3-65c1ae9c83d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:13.506838 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:13.506838 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:13.506838 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:13.506838 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:13.506838 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:13.506838 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:13.506838 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:13.506838 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:13.506838 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:13.506838 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:13.506838 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:13.506838 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:13.506838 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:13.506838 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:13.507741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:13.507741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:13.507741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:13.507741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:13.507741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:13.507741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:13.507741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:13.507741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:13.507741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:13.507741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:13.507741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:13.507741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:13.507741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:13.507741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:13.507741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:13.507741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:13.507741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:13.508877 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:13.508877 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:13.508877 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:13.508877 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:13.508877 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 374/738] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:13.508877 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3d9d67-2ae2-4e51-b3b3-65c1ae9c83d7 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:13.528254 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 365/739] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:13 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 15:13:13.529394 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc668a99-52b4-4146-b390-b5c8f9f0791a None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:13.636268 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3d9d67-2ae2-4e51-b3b3-65c1ae9c83d7 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:13.646067 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc668a99-52b4-4146-b390-b5c8f9f0791a None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:13.670682 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e3c4b-55f5-4977-8332-231a5206bb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:13.671302 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e3c4b-55f5-4977-8332-231a5206bb2b None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:13.671601 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e3c4b-55f5-4977-8332-231a5206bb2b None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:13.671884 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e3c4b-55f5-4977-8332-231a5206bb2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:13.675697 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:13.679377 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:13.682007 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980331f-5cc6-435f-9683-4ae662af4570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:13.682657 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980331f-5cc6-435f-9683-4ae662af4570 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:13.682981 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980331f-5cc6-435f-9683-4ae662af4570 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:13.683217 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980331f-5cc6-435f-9683-4ae662af4570 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:13.684746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e3c4b-55f5-4977-8332-231a5206bb2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:13.690945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e3c4b-55f5-4977-8332-231a5206bb2b None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:13.695131 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2980331f-5cc6-435f-9683-4ae662af4570 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:13:13.700032 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-2980331f-5cc6-435f-9683-4ae662af4570 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:13:13.700032 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:13.700032 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:13.700032 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:13.700032 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:13.700032 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:13.700032 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:13.700032 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:13.700032 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:13.700032 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:13.700032 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:13.700032 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:13.700032 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:13:13.700032 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:13:13.700032 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:13:13.700032 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:13:13.700946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:13:13.700946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:13:13.700946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:13:13.700946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:13:13.700946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:13.700946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:13.700946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:13.700946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:13.700946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:13.700946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:13.700946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:13.700946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:13.700946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:13.700946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:13.700946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:13.700946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:13.707927 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:13.707927 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:13.707927 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:13.707927 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:13.707927 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:13:13.707927 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:13:13.707927 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:13:13.707927 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:13:13.707927 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:13:13.707927 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:13:13.707927 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:13:13.707927 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:13.707927 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 366/740] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 15:13:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:13.707927 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980331f-5cc6-435f-9683-4ae662af4570 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:13.708788 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-ed3e3c4b-55f5-4977-8332-231a5206bb2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:13.708788 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:13.708788 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:13.708788 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:13.708788 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:13.708788 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:13.708788 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:13.708788 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:13.708788 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:13.708788 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:13.708788 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:13.708788 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:13.708788 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:13.708788 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:13.708788 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:13.709569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:13.709569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:13.709569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:13.709569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:13.709569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:13.709569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:13.709569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:13.709569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:13.709569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:13.709569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:13.709569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:13.709569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:13.709569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:13.709569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:13.709569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:13.709569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:13.709569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:13.710467 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:13.710467 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:13.710467 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:13.710467 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:13.710467 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 375/741] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:13.710467 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e3c4b-55f5-4977-8332-231a5206bb2b None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:13.806907 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980331f-5cc6-435f-9683-4ae662af4570 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:13.818549 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e3c4b-55f5-4977-8332-231a5206bb2b None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:13.844111 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8054c-8e50-4ddf-873c-6913e7f9ba70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:13.844729 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8054c-8e50-4ddf-873c-6913e7f9ba70 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:13.846517 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8054c-8e50-4ddf-873c-6913e7f9ba70 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:13.846816 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8054c-8e50-4ddf-873c-6913e7f9ba70 None admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:13.847503 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e8054c-8e50-4ddf-873c-6913e7f9ba70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:13.847810 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e8054c-8e50-4ddf-873c-6913e7f9ba70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:13.849218 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e8054c-8e50-4ddf-873c-6913e7f9ba70 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:13.856039 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a24ad5-9ce8-444f-846b-7252ede8f0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:13.856677 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a24ad5-9ce8-444f-846b-7252ede8f0c4 None admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:13.856964 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a24ad5-9ce8-444f-846b-7252ede8f0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:13.857245 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a24ad5-9ce8-444f-846b-7252ede8f0c4 None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:13.857931 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a24ad5-9ce8-444f-846b-7252ede8f0c4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:13.859286 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a24ad5-9ce8-444f-846b-7252ede8f0c4 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:13.860699 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 367/742] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:13.866027 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8054c-8e50-4ddf-873c-6913e7f9ba70 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:13.915510 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:13.940011 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 376/743] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 5 15:13:13 2022] POST /identity/v3/services => generated 240 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:13:13.941111 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a24ad5-9ce8-444f-846b-7252ede8f0c4 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:13.979672 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8054c-8e50-4ddf-873c-6913e7f9ba70 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:13.999194 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:14.045978 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3ebd2b-8bbf-4ae4-a2d4-1db3d19d76a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:14.046712 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ebd2b-8bbf-4ae4-a2d4-1db3d19d76a4 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:14.047072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ebd2b-8bbf-4ae4-a2d4-1db3d19d76a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:14.047439 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ebd2b-8bbf-4ae4-a2d4-1db3d19d76a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:14.048576 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3ebd2b-8bbf-4ae4-a2d4-1db3d19d76a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:14.048693 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3ebd2b-8bbf-4ae4-a2d4-1db3d19d76a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:14.050417 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3ebd2b-8bbf-4ae4-a2d4-1db3d19d76a4 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:14.066898 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 368/744] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:14.067968 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3ebd2b-8bbf-4ae4-a2d4-1db3d19d76a4 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:14.079294 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a24ad5-9ce8-444f-846b-7252ede8f0c4 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:14.136892 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7ae90-289e-4970-a0d2-5f70fc3c15b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:14.136892 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7ae90-289e-4970-a0d2-5f70fc3c15b2 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:14.136892 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7ae90-289e-4970-a0d2-5f70fc3c15b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:14.136892 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7ae90-289e-4970-a0d2-5f70fc3c15b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:14.137615 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7ae90-289e-4970-a0d2-5f70fc3c15b2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:14.137840 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7ae90-289e-4970-a0d2-5f70fc3c15b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:14.139234 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7ae90-289e-4970-a0d2-5f70fc3c15b2 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:14.148584 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 377/745] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 15:13:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:14.149412 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7ae90-289e-4970-a0d2-5f70fc3c15b2 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:14.198045 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3ebd2b-8bbf-4ae4-a2d4-1db3d19d76a4 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:14.208537 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:14.238491 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bf386c-986d-4720-aa02-1dbf907edec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:14.239145 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf386c-986d-4720-aa02-1dbf907edec9 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:14.239435 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf386c-986d-4720-aa02-1dbf907edec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:14.239878 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf386c-986d-4720-aa02-1dbf907edec9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:14.250907 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bf386c-986d-4720-aa02-1dbf907edec9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:14.252584 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bf386c-986d-4720-aa02-1dbf907edec9 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:14.267715 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-17bf386c-986d-4720-aa02-1dbf907edec9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 15:13:14.267715 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:14.267715 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:14.267715 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:14.267715 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:14.267715 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:14.267715 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:14.267715 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:14.267715 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:14.267715 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:14.267715 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:14.267715 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:14.267715 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 15:13:14.267715 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 15:13:14.267715 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 15:13:14.267715 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 15:13:14.268909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:14.268909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:14.268909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 15:13:14.268909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:14.268909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 15:13:14.268909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:14.268909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:14.268909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:14.268909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:14.268909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:14.268909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:14.268909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:14.268909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:14.268909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:14.268909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:14.268909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:14.268909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:14.268909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:14.270212 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:14.270212 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:14.270212 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 15:13:14.270212 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 15:13:14.270212 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 15:13:14.270212 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 15:13:14.270212 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 15:13:14.270212 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 15:13:14.270212 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 15:13:14.270212 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:14.270212 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 369/746] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 15:13:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:14.272764 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bf386c-986d-4720-aa02-1dbf907edec9 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:14.278858 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7ae90-289e-4970-a0d2-5f70fc3c15b2 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:14.305802 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:14.323413 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2136ba7-1e87-442b-8b0f-e8c8d14ebcf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:14.324377 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2136ba7-1e87-442b-8b0f-e8c8d14ebcf8 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:14.324669 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2136ba7-1e87-442b-8b0f-e8c8d14ebcf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:14.324951 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2136ba7-1e87-442b-8b0f-e8c8d14ebcf8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:14.334186 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2136ba7-1e87-442b-8b0f-e8c8d14ebcf8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:14.335930 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2136ba7-1e87-442b-8b0f-e8c8d14ebcf8 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:14.349011 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-d2136ba7-1e87-442b-8b0f-e8c8d14ebcf8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:14.349011 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:14.349011 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:14.349011 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:14.349011 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:14.349011 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:14.349011 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:14.349011 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:14.349011 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:14.349011 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:14.349011 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:14.349011 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:14.349011 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 15:13:14.349011 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 15:13:14.349011 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 15:13:14.349943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 15:13:14.349943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:14.349943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:14.349943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:14.349943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:14.349943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:14.349943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:14.349943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:14.349943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:14.349943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:14.349943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:14.349943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:14.349943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:14.349943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:14.349943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:14.349943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:14.349943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:14.350839 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 15:13:14.350839 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 15:13:14.350839 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 15:13:14.350839 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 15:13:14.350839 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 15:13:14.350839 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 15:13:14.350839 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:14.350839 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:14.353412 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 378/747] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:13:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:14.354413 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2136ba7-1e87-442b-8b0f-e8c8d14ebcf8 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:14.385640 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bf386c-986d-4720-aa02-1dbf907edec9 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:14.424659 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838bf6ad-85cd-40b9-bfc2-3daf67b4190c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:14.425823 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838bf6ad-85cd-40b9-bfc2-3daf67b4190c None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:14.426416 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838bf6ad-85cd-40b9-bfc2-3daf67b4190c None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:14.426945 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838bf6ad-85cd-40b9-bfc2-3daf67b4190c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:14.439430 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838bf6ad-85cd-40b9-bfc2-3daf67b4190c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:14.441478 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838bf6ad-85cd-40b9-bfc2-3daf67b4190c None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:14.457292 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-838bf6ad-85cd-40b9-bfc2-3daf67b4190c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:14.457292 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:14.457292 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:14.457292 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:14.457292 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:14.457292 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:14.457292 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:14.457292 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:14.457292 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:14.457292 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:14.457292 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:14.457292 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:14.457292 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:14.457292 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:14.457292 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:14.458200 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:14.458200 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:14.458200 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:14.458200 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:14.458200 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:14.458200 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:14.458200 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:14.458200 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:14.458200 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:14.458200 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:14.458200 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:14.458200 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:14.458200 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:14.458200 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:14.458200 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:14.458200 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:14.458200 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:14.459088 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:14.459088 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:14.459088 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:14.459088 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:14.465861 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 370/748] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:14.467204 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838bf6ad-85cd-40b9-bfc2-3daf67b4190c None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:14.489481 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:14.492082 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2136ba7-1e87-442b-8b0f-e8c8d14ebcf8 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:14.539398 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18c2c77-1501-4b2e-89c2-66107576130a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:14.540022 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c2c77-1501-4b2e-89c2-66107576130a None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:14.540335 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c2c77-1501-4b2e-89c2-66107576130a None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:14.540646 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c2c77-1501-4b2e-89c2-66107576130a None admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:14.541816 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18c2c77-1501-4b2e-89c2-66107576130a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:14.542485 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18c2c77-1501-4b2e-89c2-66107576130a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:14.544237 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18c2c77-1501-4b2e-89c2-66107576130a None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:14.582528 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838bf6ad-85cd-40b9-bfc2-3daf67b4190c None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:14.588218 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 379/749] 10.222.0.40 () {56 vars in 1135 bytes} [Fri Aug 5 15:13:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:14.589344 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18c2c77-1501-4b2e-89c2-66107576130a None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:14.642913 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e2f59-2383-49de-8b95-0f357b071939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:14.642913 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e2f59-2383-49de-8b95-0f357b071939 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:14.642913 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e2f59-2383-49de-8b95-0f357b071939 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:14.642913 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e2f59-2383-49de-8b95-0f357b071939 None admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:14.643698 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e2f59-2383-49de-8b95-0f357b071939 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:14.643698 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e2f59-2383-49de-8b95-0f357b071939 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:14.643974 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e2f59-2383-49de-8b95-0f357b071939 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:14.651156 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:14.669076 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 371/750] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 15:13:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:14.670436 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e2f59-2383-49de-8b95-0f357b071939 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:14.739524 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18c2c77-1501-4b2e-89c2-66107576130a None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:14.761588 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:14.778772 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d727d9d-6b8f-436d-8cad-628fde0c58c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:14.779786 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d727d9d-6b8f-436d-8cad-628fde0c58c6 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:14.780386 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d727d9d-6b8f-436d-8cad-628fde0c58c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:14.780900 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d727d9d-6b8f-436d-8cad-628fde0c58c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:14.781873 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d727d9d-6b8f-436d-8cad-628fde0c58c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:14.782383 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d727d9d-6b8f-436d-8cad-628fde0c58c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:14.784604 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d727d9d-6b8f-436d-8cad-628fde0c58c6 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:14.787663 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e2f59-2383-49de-8b95-0f357b071939 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:14.799019 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 380/751] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:14.800254 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d727d9d-6b8f-436d-8cad-628fde0c58c6 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:14.836361 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc12d6-ad23-4102-aeab-9807d885f7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:14.837006 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc12d6-ad23-4102-aeab-9807d885f7ff None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:14.837296 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc12d6-ad23-4102-aeab-9807d885f7ff None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:14.837580 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc12d6-ad23-4102-aeab-9807d885f7ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:14.906828 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:14.918482 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d727d9d-6b8f-436d-8cad-628fde0c58c6 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:14.923966 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc12d6-ad23-4102-aeab-9807d885f7ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:14.925458 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc12d6-ad23-4102-aeab-9807d885f7ff None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:14.958821 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d8f61-4841-4b59-b53d-0a031719c1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:14.960358 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d8f61-4841-4b59-b53d-0a031719c1b1 None admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:14.960861 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d8f61-4841-4b59-b53d-0a031719c1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:14.961420 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d8f61-4841-4b59-b53d-0a031719c1b1 None admin] PATH_INFO: `/v3/projects/06f27ab8886a4c0d81c4d4e3fa100a83/users/bc0497dab1f44bb7b8196d39bf01d770/roles/7c88c815db854954b516ab97cd471647` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:14.963757 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94d8f61-4841-4b59-b53d-0a031719c1b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=06f27ab8886a4c0d81c4d4e3fa100a83, user_id=bc0497dab1f44bb7b8196d39bf01d770, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:14.965696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94d8f61-4841-4b59-b53d-0a031719c1b1 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:15.015191 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:15.022350 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 381/752] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 15:13:14 2022] PUT /identity/v3/projects/06f27ab8886a4c0d81c4d4e3fa100a83/users/bc0497dab1f44bb7b8196d39bf01d770/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:13:15.024171 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d8f61-4841-4b59-b53d-0a031719c1b1 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:15.054840 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 372/753] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:13:14 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 15:13:15.054840 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9dd15-9224-451f-8a92-dbbaa80aa5da None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:15.054840 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9dd15-9224-451f-8a92-dbbaa80aa5da None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:15.054840 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9dd15-9224-451f-8a92-dbbaa80aa5da None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:15.054840 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 373/754] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:15.056436 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9dd15-9224-451f-8a92-dbbaa80aa5da None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:15.174268 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d8f61-4841-4b59-b53d-0a031719c1b1 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:15.198712 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9dd15-9224-451f-8a92-dbbaa80aa5da None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:15.222508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013d6581-e85b-4a66-8a57-d729a57499df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:15.223160 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d6581-e85b-4a66-8a57-d729a57499df None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:15.223450 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d6581-e85b-4a66-8a57-d729a57499df None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:15.224106 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d6581-e85b-4a66-8a57-d729a57499df None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:15.235648 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013d6581-e85b-4a66-8a57-d729a57499df None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:15.237159 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013d6581-e85b-4a66-8a57-d729a57499df None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:15.244955 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc9d0d-74a1-42b8-95a7-90db5662b459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:15.245909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc9d0d-74a1-42b8-95a7-90db5662b459 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:15.246417 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc9d0d-74a1-42b8-95a7-90db5662b459 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:15.246899 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc9d0d-74a1-42b8-95a7-90db5662b459 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:15.249165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-013d6581-e85b-4a66-8a57-d729a57499df None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 15:13:15.249165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:15.249165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:15.249165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:15.249165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:15.249165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:15.249165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:15.249165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:15.249165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:15.249165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:15.249165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:15.249165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:15.249165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 15:13:15.249165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 15:13:15.249165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 15:13:15.249165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 15:13:15.250098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:15.250098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:15.250098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 15:13:15.250098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:15.250098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 15:13:15.250098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:15.250098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:15.250098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:15.250098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:15.250098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:15.250098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:15.250098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:15.250098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:15.250098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:15.250098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:15.250098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:15.250098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:15.250098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:15.252055 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:15.252055 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:15.252055 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 15:13:15.252055 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 15:13:15.252055 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 15:13:15.252055 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 15:13:15.252055 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 15:13:15.252055 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 15:13:15.252055 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 15:13:15.252055 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:15.252055 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 382/755] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Aug 5 15:13:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:15.256816 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2070d-2aaa-404b-9380-7adb07d4e3b8 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:15.257114 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2070d-2aaa-404b-9380-7adb07d4e3b8 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:15.257403 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2070d-2aaa-404b-9380-7adb07d4e3b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:15.264003 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cc9d0d-74a1-42b8-95a7-90db5662b459 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:15.266836 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cc9d0d-74a1-42b8-95a7-90db5662b459 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:15.276568 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-efa2070d-2aaa-404b-9380-7adb07d4e3b8 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:15.277823 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-efa2070d-2aaa-404b-9380-7adb07d4e3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:15.281584 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:15.287337 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-f7cc9d0d-74a1-42b8-95a7-90db5662b459 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:15.287337 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:15.287337 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:15.287337 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:15.287337 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:15.287337 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:15.287337 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:15.287337 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:15.287337 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:15.287337 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:15.287337 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:15.287337 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:15.287337 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:15.287337 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:15.287337 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:15.288280 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:15.288280 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:15.288280 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:15.288280 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:15.288280 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:15.288280 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:15.288280 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:15.288280 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:15.288280 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:15.288280 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:15.288280 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:15.288280 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:15.288280 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:15.288280 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:15.288280 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:15.288280 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:15.288280 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:15.289247 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:15.289247 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:15.289247 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:15.289247 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:15.291886 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 374/756] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:15.293233 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc9d0d-74a1-42b8-95a7-90db5662b459 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:15.334880 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:15.356713 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 383/757] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:13:15.357779 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa2070d-2aaa-404b-9380-7adb07d4e3b8 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:15.439875 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc9d0d-74a1-42b8-95a7-90db5662b459 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:15.486343 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a2b84-f8d7-484e-b58d-15cb4bba08f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:15.487207 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a2b84-f8d7-484e-b58d-15cb4bba08f6 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:15.487957 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a2b84-f8d7-484e-b58d-15cb4bba08f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:15.488276 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a2b84-f8d7-484e-b58d-15cb4bba08f6 None admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:15.489395 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896a2b84-f8d7-484e-b58d-15cb4bba08f6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:15.489757 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896a2b84-f8d7-484e-b58d-15cb4bba08f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:15.490658 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa2070d-2aaa-404b-9380-7adb07d4e3b8 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:15.491194 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896a2b84-f8d7-484e-b58d-15cb4bba08f6 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:15.514491 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 375/758] 10.222.0.40 () {56 vars in 1149 bytes} [Fri Aug 5 15:13:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:15.517968 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a2b84-f8d7-484e-b58d-15cb4bba08f6 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:15.541919 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab60c712-c00b-4cb1-9f64-452c0f366182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:15.542697 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab60c712-c00b-4cb1-9f64-452c0f366182 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:15.542990 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab60c712-c00b-4cb1-9f64-452c0f366182 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:15.543316 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab60c712-c00b-4cb1-9f64-452c0f366182 None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:15.544181 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab60c712-c00b-4cb1-9f64-452c0f366182 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:15.544497 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab60c712-c00b-4cb1-9f64-452c0f366182 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:15.545965 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab60c712-c00b-4cb1-9f64-452c0f366182 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:15.556228 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:15.560920 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 384/759] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:15.561866 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab60c712-c00b-4cb1-9f64-452c0f366182 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:15.646886 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:15.659983 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a2b84-f8d7-484e-b58d-15cb4bba08f6 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:15.690527 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab60c712-c00b-4cb1-9f64-452c0f366182 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:15.705691 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca8b9ca-da99-4882-93f9-f67efaf2ca46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:15.706884 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8b9ca-da99-4882-93f9-f67efaf2ca46 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:15.707342 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8b9ca-da99-4882-93f9-f67efaf2ca46 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:15.707770 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8b9ca-da99-4882-93f9-f67efaf2ca46 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:15.708774 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca8b9ca-da99-4882-93f9-f67efaf2ca46 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:15.711026 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca8b9ca-da99-4882-93f9-f67efaf2ca46 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:15.727687 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-9ca8b9ca-da99-4882-93f9-f67efaf2ca46 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 15:13:15.727687 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:15.727687 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 15:13:15.727687 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 15:13:15.727687 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 15:13:15.727687 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 15:13:15.727687 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 15:13:15.727687 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:15.727687 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 15:13:15.727687 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:15.727687 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 15:13:15.727687 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:15.727687 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:15.727687 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:15.727687 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:15.727687 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:15.728895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:15.728895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:15.728895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:15.728895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:15.728895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:15.728895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:15.728895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:15.728895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 15:13:15.728895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 15:13:15.728895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 15:13:15.728895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 15:13:15.728895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:15.728895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:15.728895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:15.728895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:15.728895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:15.729933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:15.729933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:15.729933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:15.729933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:15.729933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:15.729933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:15.729933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:15.729933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:15.729933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:15.729933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:15.729933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:15.729933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 15:13:15.729933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:15.729933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 15:13:15.729933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:15.731746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3011b-ddef-43b7-b766-aee0e0ef1771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:15.732511 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3011b-ddef-43b7-b766-aee0e0ef1771 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:15.732854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 376/760] 10.222.0.40 () {56 vars in 1129 bytes} [Fri Aug 5 15:13:15 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:15.733973 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca8b9ca-da99-4882-93f9-f67efaf2ca46 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:15.738214 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3011b-ddef-43b7-b766-aee0e0ef1771 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:15.738600 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3011b-ddef-43b7-b766-aee0e0ef1771 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:15.810311 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:15.812750 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df3011b-ddef-43b7-b766-aee0e0ef1771 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:15.814344 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df3011b-ddef-43b7-b766-aee0e0ef1771 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:15.867859 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca8b9ca-da99-4882-93f9-f67efaf2ca46 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:15.905070 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd97cc-e058-40bf-8fd4-afee1c6a90d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:15.905860 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd97cc-e058-40bf-8fd4-afee1c6a90d0 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:15.906278 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd97cc-e058-40bf-8fd4-afee1c6a90d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:15.906655 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd97cc-e058-40bf-8fd4-afee1c6a90d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:15.914492 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cd97cc-e058-40bf-8fd4-afee1c6a90d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:15.916153 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cd97cc-e058-40bf-8fd4-afee1c6a90d0 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:15.929242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-30cd97cc-e058-40bf-8fd4-afee1c6a90d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:15.929242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:15.929242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:15.929242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:15.929242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:15.929242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:15.929242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:15.929242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:15.929242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:15.929242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:15.929242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:15.929242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:15.929242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 15:13:15.929242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 15:13:15.929242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 15:13:15.930747 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 15:13:15.930747 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:15.930747 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:15.930747 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:15.930747 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:15.930747 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:15.930747 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:15.930747 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:15.930747 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:15.930747 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:15.930747 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:15.930747 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:15.930747 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:15.930747 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:15.930747 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:15.930747 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:15.930747 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:15.931849 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 15:13:15.931849 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 15:13:15.931849 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 15:13:15.931849 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 15:13:15.931849 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 15:13:15.931849 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 15:13:15.931849 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:15.931849 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:15.936475 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 377/761] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:13:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:15.937598 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd97cc-e058-40bf-8fd4-afee1c6a90d0 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:15.954420 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 385/762] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:13:15 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 15:13:15.954420 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3011b-ddef-43b7-b766-aee0e0ef1771 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:15.959758 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:16.056216 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd97cc-e058-40bf-8fd4-afee1c6a90d0 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:16.094445 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde920d8-7a51-42f4-a699-4595f00327e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:16.096096 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde920d8-7a51-42f4-a699-4595f00327e9 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:16.096861 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde920d8-7a51-42f4-a699-4595f00327e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:16.097247 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde920d8-7a51-42f4-a699-4595f00327e9 None admin] PATH_INFO: `/v3/services` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:16.098314 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde920d8-7a51-42f4-a699-4595f00327e9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:16.098721 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde920d8-7a51-42f4-a699-4595f00327e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:16.100255 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:16.100708 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde920d8-7a51-42f4-a699-4595f00327e9 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:16.105127 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3011b-ddef-43b7-b766-aee0e0ef1771 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:16.108212 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 378/763] 10.222.0.40 () {56 vars in 1124 bytes} [Fri Aug 5 15:13:15 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:16.109183 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde920d8-7a51-42f4-a699-4595f00327e9 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:16.148155 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f75159-7bdd-47b1-a582-fecdb3199b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:16.148731 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f75159-7bdd-47b1-a582-fecdb3199b98 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:16.149018 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f75159-7bdd-47b1-a582-fecdb3199b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:16.149298 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f75159-7bdd-47b1-a582-fecdb3199b98 None admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:16.150048 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f75159-7bdd-47b1-a582-fecdb3199b98 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:16.150366 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f75159-7bdd-47b1-a582-fecdb3199b98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:16.151822 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f75159-7bdd-47b1-a582-fecdb3199b98 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:16.164498 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 386/764] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 15:13:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:16.165299 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f75159-7bdd-47b1-a582-fecdb3199b98 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:16.217818 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:16.244153 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde920d8-7a51-42f4-a699-4595f00327e9 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:16.275708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f75159-7bdd-47b1-a582-fecdb3199b98 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:16.280662 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b865c5-1286-4050-95a8-3dadd56140ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:16.281312 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b865c5-1286-4050-95a8-3dadd56140ed None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:16.281601 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b865c5-1286-4050-95a8-3dadd56140ed None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:16.281882 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b865c5-1286-4050-95a8-3dadd56140ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:16.294067 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b865c5-1286-4050-95a8-3dadd56140ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:16.295506 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b865c5-1286-4050-95a8-3dadd56140ed None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:16.310541 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7200fbaa-5a4f-4bb3-8871-d7ab9e1e8ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:16.310769 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-71b865c5-1286-4050-95a8-3dadd56140ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:16.310769 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:16.310769 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:16.310769 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:16.310769 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:16.310769 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:16.310769 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:16.310769 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:16.310769 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:16.310769 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:16.310769 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:16.310769 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:16.310769 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:16.310769 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:16.310769 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:16.311669 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:16.311669 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:16.311669 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:16.311669 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:16.311669 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:16.311669 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:16.311669 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:16.311669 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:16.311669 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:16.311669 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:16.311669 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:16.311669 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:16.311669 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:16.311669 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:16.311669 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:16.311669 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:16.311669 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:16.312605 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:16.312605 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:16.312605 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:16.312605 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:16.312828 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7200fbaa-5a4f-4bb3-8871-d7ab9e1e8ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:16.312828 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7200fbaa-5a4f-4bb3-8871-d7ab9e1e8ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:16.312828 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7200fbaa-5a4f-4bb3-8871-d7ab9e1e8ab8 None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:16.313028 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 379/765] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:16.313554 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56064a91-ab23-4fd6-bacc-c250a441348b None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:16.313744 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56064a91-ab23-4fd6-bacc-c250a441348b None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:16.313986 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56064a91-ab23-4fd6-bacc-c250a441348b None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:16.319373 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 380/766] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:16.319373 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56064a91-ab23-4fd6-bacc-c250a441348b None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:16.319664 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7200fbaa-5a4f-4bb3-8871-d7ab9e1e8ab8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:16.319664 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7200fbaa-5a4f-4bb3-8871-d7ab9e1e8ab8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:16.319664 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7200fbaa-5a4f-4bb3-8871-d7ab9e1e8ab8 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:16.328892 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 387/767] 10.222.0.40 () {56 vars in 1124 bytes} [Fri Aug 5 15:13:16 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:16.328892 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7200fbaa-5a4f-4bb3-8871-d7ab9e1e8ab8 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:16.359539 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:16.447432 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7200fbaa-5a4f-4bb3-8871-d7ab9e1e8ab8 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:16.457197 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56064a91-ab23-4fd6-bacc-c250a441348b None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:16.485562 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:16.485925 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55063fb2-d9e6-41a3-a06f-b4293b9b5a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:16.486560 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55063fb2-d9e6-41a3-a06f-b4293b9b5a15 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:16.486855 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55063fb2-d9e6-41a3-a06f-b4293b9b5a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:16.487211 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55063fb2-d9e6-41a3-a06f-b4293b9b5a15 None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:16.487832 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55063fb2-d9e6-41a3-a06f-b4293b9b5a15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:16.488178 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55063fb2-d9e6-41a3-a06f-b4293b9b5a15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:16.489557 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55063fb2-d9e6-41a3-a06f-b4293b9b5a15 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:16.494578 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fecb56f-5e00-4ef6-a9d4-c6dabeb959cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:16.498168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fecb56f-5e00-4ef6-a9d4-c6dabeb959cc None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:16.498168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fecb56f-5e00-4ef6-a9d4-c6dabeb959cc None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:16.498168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fecb56f-5e00-4ef6-a9d4-c6dabeb959cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:16.498168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fecb56f-5e00-4ef6-a9d4-c6dabeb959cc None admin] RBAC: Adding query filter params (role.id=7c88c815db854954b516ab97cd471647, scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83, user.id=85154ce7446245738edcc8702dbf6861) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:16.498168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fecb56f-5e00-4ef6-a9d4-c6dabeb959cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:16.499364 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fecb56f-5e00-4ef6-a9d4-c6dabeb959cc None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:16.502969 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 388/768] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:16.504131 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fb46b-e7e4-4ff8-922e-efcbbb3d6d5c None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:16.504525 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fb46b-e7e4-4ff8-922e-efcbbb3d6d5c None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:16.504774 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fb46b-e7e4-4ff8-922e-efcbbb3d6d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:16.520283 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 381/769] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:13:16 2022] GET /identity/v3/role_assignments?user.id=85154ce7446245738edcc8702dbf6861&scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83&role.id=7c88c815db854954b516ab97cd471647 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:16.520283 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fecb56f-5e00-4ef6-a9d4-c6dabeb959cc None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:16.535400 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-9d9fb46b-e7e4-4ff8-922e-efcbbb3d6d5c None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:16.536994 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-9d9fb46b-e7e4-4ff8-922e-efcbbb3d6d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:16.639167 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 389/770] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:13:16.640735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9fb46b-e7e4-4ff8-922e-efcbbb3d6d5c None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:16.642789 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fecb56f-5e00-4ef6-a9d4-c6dabeb959cc None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:16.661427 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:16.685399 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465031fd-0b73-472d-8c7d-4cb18f150428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:16.691152 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465031fd-0b73-472d-8c7d-4cb18f150428 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:16.691152 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465031fd-0b73-472d-8c7d-4cb18f150428 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:16.691152 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465031fd-0b73-472d-8c7d-4cb18f150428 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:16.691152 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465031fd-0b73-472d-8c7d-4cb18f150428 None admin] RBAC: Adding query filter params (interface=public, service_id=6e849ce2511b4c1faa94ecf34beee9eb, region_id=RegionOne) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:16.691152 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465031fd-0b73-472d-8c7d-4cb18f150428 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:16.691152 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465031fd-0b73-472d-8c7d-4cb18f150428 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:16.701047 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 382/771] 10.222.0.40 () {56 vars in 1251 bytes} [Fri Aug 5 15:13:16 2022] GET /identity/v3/endpoints?service_id=6e849ce2511b4c1faa94ecf34beee9eb&interface=public®ion_id=RegionOne => generated 194 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:16.702351 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465031fd-0b73-472d-8c7d-4cb18f150428 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:16.777069 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9fb46b-e7e4-4ff8-922e-efcbbb3d6d5c None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:16.804425 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:16.818322 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b5bb16-0b61-44ca-82a2-3838eb772a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:16.819142 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b5bb16-0b61-44ca-82a2-3838eb772a2b None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:16.819534 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b5bb16-0b61-44ca-82a2-3838eb772a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:16.819916 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b5bb16-0b61-44ca-82a2-3838eb772a2b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:16.827862 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b5bb16-0b61-44ca-82a2-3838eb772a2b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:16.830032 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b5bb16-0b61-44ca-82a2-3838eb772a2b None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:16.838659 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465031fd-0b73-472d-8c7d-4cb18f150428 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:16.843975 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-f1b5bb16-0b61-44ca-82a2-3838eb772a2b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:16.843975 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:16.843975 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:16.843975 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:16.843975 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:16.843975 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:16.843975 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:16.843975 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:16.843975 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:16.843975 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:16.843975 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:16.843975 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:16.843975 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 15:13:16.843975 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 15:13:16.843975 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 15:13:16.845678 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 15:13:16.845678 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:16.845678 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:16.845678 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:16.845678 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:16.845678 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:16.845678 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:16.845678 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:16.845678 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:16.845678 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:16.845678 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:16.845678 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:16.845678 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:16.845678 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:16.845678 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:16.845678 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:16.845678 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:16.846774 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 15:13:16.846774 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 15:13:16.846774 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 15:13:16.846774 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 15:13:16.846774 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 15:13:16.846774 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 15:13:16.846774 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:16.846774 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:16.848695 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 390/772] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:13:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:16.849819 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b5bb16-0b61-44ca-82a2-3838eb772a2b None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:16.876938 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd428451-9e93-4d3d-b946-d43179d7aa64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:16.877606 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd428451-9e93-4d3d-b946-d43179d7aa64 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:16.878042 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd428451-9e93-4d3d-b946-d43179d7aa64 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:16.878330 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd428451-9e93-4d3d-b946-d43179d7aa64 None admin] PATH_INFO: `/v3/services/network` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:16.879122 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd428451-9e93-4d3d-b946-d43179d7aa64 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:16.880575 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd428451-9e93-4d3d-b946-d43179d7aa64 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:16.890940 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-cd428451-9e93-4d3d-b946-d43179d7aa64 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 15:13:16.890940 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:16.890940 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 15:13:16.890940 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 15:13:16.890940 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 15:13:16.890940 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 15:13:16.890940 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 15:13:16.890940 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:16.890940 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 15:13:16.890940 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:16.890940 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 15:13:16.890940 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:16.890940 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:16.890940 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:16.890940 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:16.890940 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:16.890940 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:16.892614 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:16.892614 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:16.892614 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:16.892614 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:16.892614 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:16.892614 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:16.892614 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 15:13:16.892614 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 15:13:16.892614 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 15:13:16.892614 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 15:13:16.892614 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:16.892614 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:16.892614 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:16.892614 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:16.892614 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:16.892614 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:16.893628 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:16.893628 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:16.893628 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:16.893628 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:16.893628 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:16.893628 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:16.893628 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:16.893628 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:16.893628 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:16.893628 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:16.893628 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 15:13:16.893628 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:16.893628 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 15:13:16.893628 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:16.897673 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 383/773] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:13:16 2022] GET /identity/v3/services/network => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:16.898723 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd428451-9e93-4d3d-b946-d43179d7aa64 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:16.938065 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:16.968701 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b5bb16-0b61-44ca-82a2-3838eb772a2b None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:17.009744 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17168fde-a1fe-4426-8208-065df5dbcb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:17.010555 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17168fde-a1fe-4426-8208-065df5dbcb94 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:17.010950 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17168fde-a1fe-4426-8208-065df5dbcb94 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:17.011340 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17168fde-a1fe-4426-8208-065df5dbcb94 None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:17.011789 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd428451-9e93-4d3d-b946-d43179d7aa64 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:17.012293 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17168fde-a1fe-4426-8208-065df5dbcb94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:17.013924 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17168fde-a1fe-4426-8208-065df5dbcb94 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:17.028100 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 391/774] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Aug 5 15:13:16 2022] GET /identity/v3/services => generated 772 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:17.028100 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17168fde-a1fe-4426-8208-065df5dbcb94 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:17.061070 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:17.092262 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b460271-a35c-4b80-aa38-ed7d17c64473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:17.093820 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b460271-a35c-4b80-aa38-ed7d17c64473 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:17.094267 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b460271-a35c-4b80-aa38-ed7d17c64473 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:17.094755 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b460271-a35c-4b80-aa38-ed7d17c64473 None admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:17.095761 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b460271-a35c-4b80-aa38-ed7d17c64473 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:17.096286 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b460271-a35c-4b80-aa38-ed7d17c64473 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:17.097918 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b460271-a35c-4b80-aa38-ed7d17c64473 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:17.120257 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 384/775] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 15:13:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:17.126263 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b460271-a35c-4b80-aa38-ed7d17c64473 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:17.158053 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17168fde-a1fe-4426-8208-065df5dbcb94 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:17.198595 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b21621-e431-4bd8-a6e3-5aa3e5dcaafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:17.199349 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b21621-e431-4bd8-a6e3-5aa3e5dcaafb None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:17.199724 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b21621-e431-4bd8-a6e3-5aa3e5dcaafb None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:17.200094 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b21621-e431-4bd8-a6e3-5aa3e5dcaafb None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:17.200896 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b21621-e431-4bd8-a6e3-5aa3e5dcaafb None admin] RBAC: Adding query filter params (name=network) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:17.201281 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b21621-e431-4bd8-a6e3-5aa3e5dcaafb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:17.202739 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b21621-e431-4bd8-a6e3-5aa3e5dcaafb None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:17.215460 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 392/776] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 5 15:13:17 2022] GET /identity/v3/services?name=network => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:17.222874 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b21621-e431-4bd8-a6e3-5aa3e5dcaafb None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:17.226394 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:17.291314 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b460271-a35c-4b80-aa38-ed7d17c64473 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:17.333226 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2db6cb-891f-4de3-8edf-2b57581fde37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:17.334009 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2db6cb-891f-4de3-8edf-2b57581fde37 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:17.334305 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2db6cb-891f-4de3-8edf-2b57581fde37 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:17.334590 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2db6cb-891f-4de3-8edf-2b57581fde37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:17.335393 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2db6cb-891f-4de3-8edf-2b57581fde37 None admin] RBAC: Adding query filter params (role.id=266a023f66e043f99efe2941fe58d481, scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83, user.id=1654fb6a81e14d6b99ab495bb6d029d7) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:17.335713 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2db6cb-891f-4de3-8edf-2b57581fde37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:17.337179 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2db6cb-891f-4de3-8edf-2b57581fde37 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:17.343288 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 385/777] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:13:17 2022] GET /identity/v3/role_assignments?user.id=1654fb6a81e14d6b99ab495bb6d029d7&scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83&role.id=266a023f66e043f99efe2941fe58d481 => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:17.344584 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5357b8-b74c-43b6-8f38-e14f004ea88d None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:17.344930 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5357b8-b74c-43b6-8f38-e14f004ea88d None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:17.345219 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5357b8-b74c-43b6-8f38-e14f004ea88d None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:17.346825 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 386/778] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:17.355588 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff4c0d-ac14-4507-988b-a9107819c19f None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:17.355588 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff4c0d-ac14-4507-988b-a9107819c19f None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:17.355588 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff4c0d-ac14-4507-988b-a9107819c19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:17.361949 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b21621-e431-4bd8-a6e3-5aa3e5dcaafb None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:17.385484 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-d0ff4c0d-ac14-4507-988b-a9107819c19f None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:17.386734 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-d0ff4c0d-ac14-4507-988b-a9107819c19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:17.440122 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a79f86-b262-4755-8741-9bfafa7f6f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:17.440723 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a79f86-b262-4755-8741-9bfafa7f6f98 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:17.441009 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a79f86-b262-4755-8741-9bfafa7f6f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:17.441286 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a79f86-b262-4755-8741-9bfafa7f6f98 None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:17.441994 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a79f86-b262-4755-8741-9bfafa7f6f98 None admin] RBAC: Adding query filter params (type=network) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:17.442303 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a79f86-b262-4755-8741-9bfafa7f6f98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:17.443696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a79f86-b262-4755-8741-9bfafa7f6f98 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:17.457712 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 393/779] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 5 15:13:17 2022] GET /identity/v3/services?type=network => generated 350 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:17.463000 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a79f86-b262-4755-8741-9bfafa7f6f98 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:17.494923 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:17.533521 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 387/780] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 15:13:17.558956 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:17.581534 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff4c0d-ac14-4507-988b-a9107819c19f None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:17.594684 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a79f86-b262-4755-8741-9bfafa7f6f98 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:17.629965 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea9ec1c-2729-4ceb-a4b3-ba455655d1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:17.630599 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9ec1c-2729-4ceb-a4b3-ba455655d1d9 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:17.630884 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9ec1c-2729-4ceb-a4b3-ba455655d1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:17.631162 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9ec1c-2729-4ceb-a4b3-ba455655d1d9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:17.631934 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea9ec1c-2729-4ceb-a4b3-ba455655d1d9 None admin] RBAC: Adding query filter params (interface=public, service_id=7a4a49ad57e647738840bcb74497ec88, region_id=RegionOne) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:17.632279 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea9ec1c-2729-4ceb-a4b3-ba455655d1d9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:17.633654 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea9ec1c-2729-4ceb-a4b3-ba455655d1d9 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:17.642256 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 394/781] 10.222.0.40 () {56 vars in 1251 bytes} [Fri Aug 5 15:13:17 2022] GET /identity/v3/endpoints?service_id=7a4a49ad57e647738840bcb74497ec88&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:17.648232 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea9ec1c-2729-4ceb-a4b3-ba455655d1d9 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:17.754245 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff4c0d-ac14-4507-988b-a9107819c19f None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:17.770671 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea9ec1c-2729-4ceb-a4b3-ba455655d1d9 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:17.800498 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:17.809741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f748261-55e0-427b-9b89-b54a17e7a1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:17.810381 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f748261-55e0-427b-9b89-b54a17e7a1ac None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:17.810669 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f748261-55e0-427b-9b89-b54a17e7a1ac None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:17.810991 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f748261-55e0-427b-9b89-b54a17e7a1ac None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:17.811624 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f748261-55e0-427b-9b89-b54a17e7a1ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:17.813095 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f748261-55e0-427b-9b89-b54a17e7a1ac None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:17.822631 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 395/782] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Aug 5 15:13:17 2022] GET /identity/v3/services => generated 772 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:17.831287 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af971756-9ef1-4850-999c-caaae88c2c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:17.831960 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af971756-9ef1-4850-999c-caaae88c2c79 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:17.832279 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af971756-9ef1-4850-999c-caaae88c2c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:17.832560 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af971756-9ef1-4850-999c-caaae88c2c79 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:17.844665 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af971756-9ef1-4850-999c-caaae88c2c79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:13:17.850309 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-af971756-9ef1-4850-999c-caaae88c2c79 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:13:17.850309 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:17.850309 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:17.850309 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:17.850309 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:17.850309 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:17.850309 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:17.850309 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:17.850309 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:17.850309 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:17.850309 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:17.850309 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:17.850309 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:13:17.850309 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:13:17.850309 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:13:17.850309 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:13:17.851273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:13:17.851273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:13:17.851273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:13:17.851273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:13:17.851273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:17.851273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:17.851273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:17.851273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:17.851273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:17.851273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:17.851273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:17.851273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:17.851273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:17.851273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:17.851273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:17.851273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:17.852167 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:17.852167 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:17.852167 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:17.852167 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:17.852167 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:13:17.852167 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:13:17.852167 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:13:17.852167 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:13:17.852167 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:13:17.852167 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:13:17.852167 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:13:17.852167 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:17.852167 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 388/783] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 15:13:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:13:17.858043 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f748261-55e0-427b-9b89-b54a17e7a1ac None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:17.977186 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f748261-55e0-427b-9b89-b54a17e7a1ac None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:18.018631 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d0d936-067c-4bc8-8d32-8eea8f32221f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:18.019309 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d0d936-067c-4bc8-8d32-8eea8f32221f None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:18.019596 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d0d936-067c-4bc8-8d32-8eea8f32221f None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:18.019925 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d0d936-067c-4bc8-8d32-8eea8f32221f None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:18.020708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d0d936-067c-4bc8-8d32-8eea8f32221f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:18.021019 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d0d936-067c-4bc8-8d32-8eea8f32221f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:18.022391 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d0d936-067c-4bc8-8d32-8eea8f32221f None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:18.031523 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 396/784] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:13:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:18.041186 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af971756-9ef1-4850-999c-caaae88c2c79 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:18.088207 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:18.162942 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af971756-9ef1-4850-999c-caaae88c2c79 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:18.221873 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b4ec6-4976-4027-8fd3-fc743b6711c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:18.222515 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b4ec6-4976-4027-8fd3-fc743b6711c2 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:18.222799 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b4ec6-4976-4027-8fd3-fc743b6711c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:18.223076 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b4ec6-4976-4027-8fd3-fc743b6711c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:18.238393 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2b4ec6-4976-4027-8fd3-fc743b6711c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:18.239819 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2b4ec6-4976-4027-8fd3-fc743b6711c2 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:18.255372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-1c2b4ec6-4976-4027-8fd3-fc743b6711c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:18.255372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:18.255372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:18.255372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:18.255372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:18.255372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:18.255372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:18.255372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:18.255372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:18.255372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:18.255372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:18.255372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:18.255372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:18.255372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:18.255372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:18.256283 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:18.256283 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:18.256283 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:18.256283 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:18.256283 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:18.256283 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:18.256283 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:18.256283 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:18.256283 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:18.256283 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:18.256283 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:18.256283 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:18.256283 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:18.256283 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:18.256283 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:18.256283 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:18.256283 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:18.257182 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:18.257182 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:18.257182 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:18.257182 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:18.260168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 389/785] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:18.262597 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d0d936-067c-4bc8-8d32-8eea8f32221f None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:18.320636 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:18.389774 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d0d936-067c-4bc8-8d32-8eea8f32221f None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:18.440688 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f914401-3fb7-451c-97c6-6b868997de35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:18.441318 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f914401-3fb7-451c-97c6-6b868997de35 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:18.441601 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f914401-3fb7-451c-97c6-6b868997de35 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:18.441880 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f914401-3fb7-451c-97c6-6b868997de35 None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:18.442577 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f914401-3fb7-451c-97c6-6b868997de35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:18.442887 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f914401-3fb7-451c-97c6-6b868997de35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:18.444271 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f914401-3fb7-451c-97c6-6b868997de35 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:18.461244 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 397/786] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:18.466751 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b4ec6-4976-4027-8fd3-fc743b6711c2 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:18.568260 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:18.598409 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b4ec6-4976-4027-8fd3-fc743b6711c2 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:18.651464 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e4fe6-495d-4564-a6e3-70b26d2f5c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:18.652221 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e4fe6-495d-4564-a6e3-70b26d2f5c9a None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:18.652511 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e4fe6-495d-4564-a6e3-70b26d2f5c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:18.652846 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e4fe6-495d-4564-a6e3-70b26d2f5c9a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:18.662035 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1e4fe6-495d-4564-a6e3-70b26d2f5c9a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:18.663480 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1e4fe6-495d-4564-a6e3-70b26d2f5c9a None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:18.675943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-4a1e4fe6-495d-4564-a6e3-70b26d2f5c9a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 15:13:18.675943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:18.675943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:18.675943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:18.675943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:18.675943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:18.675943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:18.675943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:18.675943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:18.675943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:18.675943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:18.675943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:18.675943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 15:13:18.675943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 15:13:18.675943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 15:13:18.675943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 15:13:18.678110 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:18.678110 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:18.678110 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 15:13:18.678110 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:18.678110 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 15:13:18.678110 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:18.678110 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:18.678110 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:18.678110 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:18.678110 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:18.678110 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:18.678110 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:18.678110 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:18.678110 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:18.678110 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:18.678110 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:18.678110 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:18.678110 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:18.679594 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:18.679594 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:18.679594 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 15:13:18.679594 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 15:13:18.679594 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 15:13:18.679594 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 15:13:18.679594 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 15:13:18.679594 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 15:13:18.679594 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 15:13:18.679594 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:18.679594 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 390/787] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Aug 5 15:13:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:18.683269 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f914401-3fb7-451c-97c6-6b868997de35 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:18.846887 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f914401-3fb7-451c-97c6-6b868997de35 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:18.871413 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:18.893658 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9564e8-bf9b-43d2-b307-5e3b6009d56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:18.894503 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9564e8-bf9b-43d2-b307-5e3b6009d56e None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:18.894935 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9564e8-bf9b-43d2-b307-5e3b6009d56e None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:18.895443 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9564e8-bf9b-43d2-b307-5e3b6009d56e None admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:18.896415 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9564e8-bf9b-43d2-b307-5e3b6009d56e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:18.896897 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9564e8-bf9b-43d2-b307-5e3b6009d56e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:18.898553 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9564e8-bf9b-43d2-b307-5e3b6009d56e None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:18.926587 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 398/788] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Aug 5 15:13:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:18.932445 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e4fe6-495d-4564-a6e3-70b26d2f5c9a None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:19.079932 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e4fe6-495d-4564-a6e3-70b26d2f5c9a None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:19.130772 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3b1d4-9c9b-4790-a997-601e675514a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:19.131319 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3b1d4-9c9b-4790-a997-601e675514a5 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:19.131606 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3b1d4-9c9b-4790-a997-601e675514a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:19.131880 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3b1d4-9c9b-4790-a997-601e675514a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:19.153096 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:19.246163 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d3b1d4-9c9b-4790-a997-601e675514a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:19.247890 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d3b1d4-9c9b-4790-a997-601e675514a5 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:19.315156 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60d7f9-70d2-4c11-a83a-8edbdb7f4790 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:19.315689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60d7f9-70d2-4c11-a83a-8edbdb7f4790 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:19.315689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60d7f9-70d2-4c11-a83a-8edbdb7f4790 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:19.317437 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 399/789] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:19.323137 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccd72e-25da-439e-88d2-be1057058418 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:19.323776 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccd72e-25da-439e-88d2-be1057058418 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:19.324399 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccd72e-25da-439e-88d2-be1057058418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:19.349750 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-21ccd72e-25da-439e-88d2-be1057058418 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:19.356649 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-21ccd72e-25da-439e-88d2-be1057058418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:19.403609 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 391/790] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:13:18 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 15:13:19.411051 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3b1d4-9c9b-4790-a997-601e675514a5 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:19.483467 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 400/791] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 15:13:19.486648 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:19.519350 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ccd72e-25da-439e-88d2-be1057058418 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:19.584388 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3b1d4-9c9b-4790-a997-601e675514a5 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:19.649012 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eda2f0-f063-4dbc-9d6a-ea6ef7d22ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:19.650002 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eda2f0-f063-4dbc-9d6a-ea6ef7d22ef8 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:19.650480 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eda2f0-f063-4dbc-9d6a-ea6ef7d22ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:19.650976 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eda2f0-f063-4dbc-9d6a-ea6ef7d22ef8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:19.671443 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eda2f0-f063-4dbc-9d6a-ea6ef7d22ef8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:19.674270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eda2f0-f063-4dbc-9d6a-ea6ef7d22ef8 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:19.687860 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ccd72e-25da-439e-88d2-be1057058418 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:19.692024 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:19.696602 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-44eda2f0-f063-4dbc-9d6a-ea6ef7d22ef8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:19.696602 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:19.696602 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:19.696602 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:19.696602 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:19.696602 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:19.696602 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:19.696602 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:19.696602 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:19.696602 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:19.696602 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:19.696602 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:19.696602 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:19.696602 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:19.696602 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:19.697483 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:19.697483 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:19.697483 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:19.697483 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:19.697483 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:19.697483 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:19.697483 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:19.697483 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:19.697483 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:19.697483 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:19.697483 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:19.697483 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:19.697483 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:19.697483 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:19.697483 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:19.697483 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:19.697483 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:19.698364 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:19.698364 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:19.698364 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:19.698364 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:19.700959 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 392/792] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:13:19.703101 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ab976-b61a-40df-99db-2e40af639e82 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:19.703696 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ab976-b61a-40df-99db-2e40af639e82 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:19.704866 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ab976-b61a-40df-99db-2e40af639e82 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:19.707176 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 393/793] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:19.708148 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ab976-b61a-40df-99db-2e40af639e82 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:19.733472 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8c69c-843a-4523-8001-dd78b3452442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:19.734217 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8c69c-843a-4523-8001-dd78b3452442 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:19.734573 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8c69c-843a-4523-8001-dd78b3452442 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:19.734926 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8c69c-843a-4523-8001-dd78b3452442 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:19.749029 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c8c69c-843a-4523-8001-dd78b3452442 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:13:19.755955 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-72c8c69c-843a-4523-8001-dd78b3452442 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:13:19.755955 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:19.755955 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:19.755955 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:19.755955 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:19.755955 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:19.755955 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:19.755955 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:19.755955 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:19.755955 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:19.755955 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:19.755955 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:19.755955 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:13:19.755955 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:13:19.755955 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:13:19.755955 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:13:19.756914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:13:19.756914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:13:19.756914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:13:19.756914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:13:19.756914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:19.756914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:19.756914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:19.756914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:19.756914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:19.756914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:19.756914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:19.756914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:19.756914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:19.756914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:19.756914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:19.756914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:19.757737 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:19.757737 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:19.757737 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:19.757737 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:19.757737 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:13:19.757737 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:13:19.757737 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:13:19.757737 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:13:19.757737 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:13:19.757737 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:13:19.757737 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:13:19.757737 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:19.757737 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 401/794] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 15:13:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:13:19.760904 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3084491-fac6-43f4-b229-f7eea9443807 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:19.761203 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3084491-fac6-43f4-b229-f7eea9443807 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:19.761495 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3084491-fac6-43f4-b229-f7eea9443807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:19.781220 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-a3084491-fac6-43f4-b229-f7eea9443807 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:19.782585 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-a3084491-fac6-43f4-b229-f7eea9443807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:19.822976 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:19.827676 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ab976-b61a-40df-99db-2e40af639e82 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:19.853749 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 402/795] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:13:19.857780 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1acaf23-8cbe-4440-8313-8d91f5176f54 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:19.858655 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1acaf23-8cbe-4440-8313-8d91f5176f54 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:19.859389 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1acaf23-8cbe-4440-8313-8d91f5176f54 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:19.862208 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 403/796] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:19.863261 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1acaf23-8cbe-4440-8313-8d91f5176f54 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:19.863600 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5987e1b4-9984-4191-8adb-3d10a0b8293c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:19.864242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5987e1b4-9984-4191-8adb-3d10a0b8293c None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:19.864540 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5987e1b4-9984-4191-8adb-3d10a0b8293c None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:19.864807 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5987e1b4-9984-4191-8adb-3d10a0b8293c None admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:19.865573 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5987e1b4-9984-4191-8adb-3d10a0b8293c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:19.865888 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5987e1b4-9984-4191-8adb-3d10a0b8293c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:19.867253 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5987e1b4-9984-4191-8adb-3d10a0b8293c None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:19.881144 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 394/797] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:19.882171 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3904d2ff-8b59-472e-aef2-5154a6d732e5 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:19.882549 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3904d2ff-8b59-472e-aef2-5154a6d732e5 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:19.882739 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3904d2ff-8b59-472e-aef2-5154a6d732e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:19.901789 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-3904d2ff-8b59-472e-aef2-5154a6d732e5 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:19.902989 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-3904d2ff-8b59-472e-aef2-5154a6d732e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:19.973388 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 395/798] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:13:19.974228 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3904d2ff-8b59-472e-aef2-5154a6d732e5 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:19.978467 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:19.988672 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1acaf23-8cbe-4440-8313-8d91f5176f54 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:20.029077 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e422c-fe50-42ec-860c-12a20b2493a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:20.029753 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e422c-fe50-42ec-860c-12a20b2493a5 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:20.030038 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e422c-fe50-42ec-860c-12a20b2493a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:20.030316 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e422c-fe50-42ec-860c-12a20b2493a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:20.031044 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e422c-fe50-42ec-860c-12a20b2493a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:20.031356 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e422c-fe50-42ec-860c-12a20b2493a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:20.032785 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e422c-fe50-42ec-860c-12a20b2493a5 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:20.041585 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 404/799] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:13:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:20.046674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e422c-fe50-42ec-860c-12a20b2493a5 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:20.067767 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:20.080155 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3904d2ff-8b59-472e-aef2-5154a6d732e5 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:20.120877 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4284eb-00a0-4eac-af56-03853c957efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:20.121551 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4284eb-00a0-4eac-af56-03853c957efb None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:20.121837 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4284eb-00a0-4eac-af56-03853c957efb None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:20.122115 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4284eb-00a0-4eac-af56-03853c957efb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:20.133964 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4284eb-00a0-4eac-af56-03853c957efb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:13:20.140006 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-7c4284eb-00a0-4eac-af56-03853c957efb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:13:20.140006 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:20.140006 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:20.140006 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:20.140006 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:20.140006 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:20.140006 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:20.140006 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:20.140006 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:20.140006 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:20.140006 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:20.140006 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:20.140006 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:13:20.140006 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:13:20.140006 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:13:20.140006 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:13:20.141063 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:13:20.141063 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:13:20.141063 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:13:20.141063 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:13:20.141063 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:20.141063 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:20.141063 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:20.141063 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:20.141063 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:20.141063 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:20.141063 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:20.141063 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:20.141063 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:20.141063 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:20.141063 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:20.141063 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:20.141952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:20.141952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:20.141952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:20.141952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:20.141952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:13:20.141952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:13:20.141952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:13:20.141952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:13:20.141952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:13:20.141952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:13:20.141952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:13:20.141952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:20.141952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 396/800] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 15:13:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:20.141952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4284eb-00a0-4eac-af56-03853c957efb None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:20.163471 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e422c-fe50-42ec-860c-12a20b2493a5 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:20.198925 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a12356-e650-43b7-9ef8-0c6fe21a64bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:20.199640 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a12356-e650-43b7-9ef8-0c6fe21a64bf None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:20.199991 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a12356-e650-43b7-9ef8-0c6fe21a64bf None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:20.200305 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a12356-e650-43b7-9ef8-0c6fe21a64bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:20.207160 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a12356-e650-43b7-9ef8-0c6fe21a64bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:20.208660 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a12356-e650-43b7-9ef8-0c6fe21a64bf None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:20.220708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-56a12356-e650-43b7-9ef8-0c6fe21a64bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:20.220708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:20.220708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:20.220708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:20.220708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:20.220708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:20.220708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:20.220708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:20.220708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:20.220708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:20.220708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:20.220708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:20.220708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 15:13:20.220708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 15:13:20.220708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 15:13:20.221607 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 15:13:20.221607 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:20.221607 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:20.221607 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:20.221607 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:20.221607 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:20.221607 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:20.221607 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:20.221607 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:20.221607 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:20.221607 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:20.221607 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:20.221607 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:20.221607 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:20.221607 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:20.221607 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:20.221607 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:20.222583 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 15:13:20.222583 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 15:13:20.222583 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 15:13:20.222583 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 15:13:20.222583 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 15:13:20.222583 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 15:13:20.222583 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:20.222583 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:20.222583 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 405/801] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:13:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:20.223075 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a12356-e650-43b7-9ef8-0c6fe21a64bf None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:20.244575 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:20.258657 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4284eb-00a0-4eac-af56-03853c957efb None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:20.292109 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d877d4-ec3f-413b-ac2d-c241a3c1f806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:20.292743 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d877d4-ec3f-413b-ac2d-c241a3c1f806 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:20.293029 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d877d4-ec3f-413b-ac2d-c241a3c1f806 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:20.293302 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d877d4-ec3f-413b-ac2d-c241a3c1f806 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:20.293924 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d877d4-ec3f-413b-ac2d-c241a3c1f806 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:20.295287 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d877d4-ec3f-413b-ac2d-c241a3c1f806 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:20.304602 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-13d877d4-ec3f-413b-ac2d-c241a3c1f806 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 15:13:20.304602 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:20.304602 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 15:13:20.304602 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 15:13:20.304602 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 15:13:20.304602 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 15:13:20.304602 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 15:13:20.304602 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:20.304602 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 15:13:20.304602 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:20.304602 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 15:13:20.304602 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:20.304602 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:20.304602 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:20.304602 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:20.304602 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:20.305728 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:20.305728 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:20.305728 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:20.305728 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:20.305728 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:20.305728 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:20.305728 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:20.305728 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 15:13:20.305728 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 15:13:20.305728 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 15:13:20.305728 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 15:13:20.305728 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:20.305728 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:20.305728 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:20.305728 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:20.305728 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:20.306899 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:20.306899 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:20.306899 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:20.306899 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:20.306899 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:20.306899 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:20.306899 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:20.306899 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:20.306899 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:20.306899 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:20.306899 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:20.306899 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 15:13:20.306899 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:20.306899 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 15:13:20.306899 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:20.306899 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 397/802] 10.222.0.40 () {56 vars in 1129 bytes} [Fri Aug 5 15:13:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:20.307938 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d877d4-ec3f-413b-ac2d-c241a3c1f806 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:20.320895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:20.327364 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a12356-e650-43b7-9ef8-0c6fe21a64bf None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:20.361095 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aaf565-4cba-4669-a609-d5adf08dbf6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:20.361735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aaf565-4cba-4669-a609-d5adf08dbf6c None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:20.362022 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aaf565-4cba-4669-a609-d5adf08dbf6c None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:20.362299 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aaf565-4cba-4669-a609-d5adf08dbf6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:20.373004 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aaf565-4cba-4669-a609-d5adf08dbf6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:20.374475 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aaf565-4cba-4669-a609-d5adf08dbf6c None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:20.388689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-89aaf565-4cba-4669-a609-d5adf08dbf6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:20.388689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:20.388689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:20.388689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:20.388689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:20.388689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:20.388689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:20.388689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:20.388689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:20.388689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:20.388689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:20.388689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:20.388689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:20.388689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:20.388689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:20.389601 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:20.389601 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:20.389601 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:20.389601 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:20.389601 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:20.389601 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:20.389601 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:20.389601 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:20.389601 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:20.389601 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:20.389601 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:20.389601 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:20.389601 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:20.389601 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:20.389601 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:20.389601 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:20.389601 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:20.390492 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:20.390492 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:20.390492 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:20.390492 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:20.390492 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 406/803] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:20.392445 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5942f-72c5-4238-95a9-496f27b4296c None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:20.392852 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5942f-72c5-4238-95a9-496f27b4296c None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:20.393132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5942f-72c5-4238-95a9-496f27b4296c None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:20.394523 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 407/804] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:20.397143 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5942f-72c5-4238-95a9-496f27b4296c None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:20.405878 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d877d4-ec3f-413b-ac2d-c241a3c1f806 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:20.440554 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c94c0-1201-48fd-840b-e7d0098ba3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:20.441163 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c94c0-1201-48fd-840b-e7d0098ba3c1 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:20.441448 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c94c0-1201-48fd-840b-e7d0098ba3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:20.441809 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c94c0-1201-48fd-840b-e7d0098ba3c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:20.442528 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c94c0-1201-48fd-840b-e7d0098ba3c1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:20.442838 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c94c0-1201-48fd-840b-e7d0098ba3c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:20.444266 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c94c0-1201-48fd-840b-e7d0098ba3c1 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:20.452006 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 398/805] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 15:13:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:20.452802 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c94c0-1201-48fd-840b-e7d0098ba3c1 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:20.465264 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:20.500362 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5942f-72c5-4238-95a9-496f27b4296c None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:20.529127 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:20.534587 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a5e9f7-d81f-4b83-bc87-ab2709e4a9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:20.535203 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5e9f7-d81f-4b83-bc87-ab2709e4a9c1 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:20.535488 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5e9f7-d81f-4b83-bc87-ab2709e4a9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:20.535776 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5e9f7-d81f-4b83-bc87-ab2709e4a9c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:20.536567 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a5e9f7-d81f-4b83-bc87-ab2709e4a9c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:20.536879 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a5e9f7-d81f-4b83-bc87-ab2709e4a9c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:20.538350 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a5e9f7-d81f-4b83-bc87-ab2709e4a9c1 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:20.551077 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 408/806] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 15:13:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:20.552301 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a5e9f7-d81f-4b83-bc87-ab2709e4a9c1 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:20.554824 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c94c0-1201-48fd-840b-e7d0098ba3c1 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:20.588369 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad67a39d-f768-4470-8cfa-4d2c3ba4d3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:20.588978 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad67a39d-f768-4470-8cfa-4d2c3ba4d3f1 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:20.589260 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad67a39d-f768-4470-8cfa-4d2c3ba4d3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:20.589531 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad67a39d-f768-4470-8cfa-4d2c3ba4d3f1 None admin] PATH_INFO: `/v3/services` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:20.590195 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad67a39d-f768-4470-8cfa-4d2c3ba4d3f1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:20.590500 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad67a39d-f768-4470-8cfa-4d2c3ba4d3f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:20.591837 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad67a39d-f768-4470-8cfa-4d2c3ba4d3f1 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:20.596836 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 399/807] 10.222.0.40 () {56 vars in 1124 bytes} [Fri Aug 5 15:13:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:20.597944 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a68e2c-8158-4cc1-8d8c-0cbbe3583aab None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:20.598231 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a68e2c-8158-4cc1-8d8c-0cbbe3583aab None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:20.598510 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a68e2c-8158-4cc1-8d8c-0cbbe3583aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:20.615529 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-d4a68e2c-8158-4cc1-8d8c-0cbbe3583aab None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:20.616736 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-d4a68e2c-8158-4cc1-8d8c-0cbbe3583aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:20.655136 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a5e9f7-d81f-4b83-bc87-ab2709e4a9c1 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:20.681266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:20.682767 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 400/808] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:13:20.683592 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a68e2c-8158-4cc1-8d8c-0cbbe3583aab None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:20.688792 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a695c-c058-4edf-a38f-67d61911dff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:20.694684 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a695c-c058-4edf-a38f-67d61911dff9 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:20.694684 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a695c-c058-4edf-a38f-67d61911dff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:20.694684 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a695c-c058-4edf-a38f-67d61911dff9 None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:20.694684 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158a695c-c058-4edf-a38f-67d61911dff9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:20.694684 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158a695c-c058-4edf-a38f-67d61911dff9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:20.694684 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158a695c-c058-4edf-a38f-67d61911dff9 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:20.705063 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 409/809] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:20.706102 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a695c-c058-4edf-a38f-67d61911dff9 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:20.754602 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:20.787922 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a68e2c-8158-4cc1-8d8c-0cbbe3583aab None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:20.806799 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a695c-c058-4edf-a38f-67d61911dff9 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:20.821413 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c425ee-534d-4cfb-91ce-6c72c30c0098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:20.822052 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c425ee-534d-4cfb-91ce-6c72c30c0098 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:20.822335 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c425ee-534d-4cfb-91ce-6c72c30c0098 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:20.822609 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c425ee-534d-4cfb-91ce-6c72c30c0098 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:20.832919 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c425ee-534d-4cfb-91ce-6c72c30c0098 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:20.834356 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c425ee-534d-4cfb-91ce-6c72c30c0098 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:20.840006 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5291e1-c2b1-4896-9afe-21ca0e474def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:20.840641 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5291e1-c2b1-4896-9afe-21ca0e474def None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:20.840931 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5291e1-c2b1-4896-9afe-21ca0e474def None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:20.841213 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5291e1-c2b1-4896-9afe-21ca0e474def None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:20.842005 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5291e1-c2b1-4896-9afe-21ca0e474def None admin] RBAC: Adding query filter params (role.id=7c88c815db854954b516ab97cd471647, scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83, user.id=bc0497dab1f44bb7b8196d39bf01d770) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:20.842320 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5291e1-c2b1-4896-9afe-21ca0e474def None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:20.843744 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5291e1-c2b1-4896-9afe-21ca0e474def None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:20.849773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-38c425ee-534d-4cfb-91ce-6c72c30c0098 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:20.849773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:20.849773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:20.849773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:20.849773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:20.849773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:20.849773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:20.849773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:20.849773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:20.849773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:20.849773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:20.849773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:20.849773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:20.849773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:20.849773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:20.850642 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:20.850642 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:20.850642 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:20.850642 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:20.850642 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:20.850642 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:20.850642 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:20.850642 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:20.850642 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:20.850642 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:20.850642 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:20.850642 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:20.850642 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:20.850642 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:20.850642 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:20.850642 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:20.850642 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:20.851522 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:20.851522 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:20.851522 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:20.851522 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:20.851751 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 410/810] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:13:20 2022] GET /identity/v3/role_assignments?user.id=bc0497dab1f44bb7b8196d39bf01d770&scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83&role.id=7c88c815db854954b516ab97cd471647 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:20.851751 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5291e1-c2b1-4896-9afe-21ca0e474def None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:20.856909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 401/811] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:20.856909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c425ee-534d-4cfb-91ce-6c72c30c0098 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:20.915409 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:20.968711 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c425ee-534d-4cfb-91ce-6c72c30c0098 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:20.972172 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5291e1-c2b1-4896-9afe-21ca0e474def None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:20.991058 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:21.009342 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1588e6-b8b0-407d-9bd6-ff3843d17394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:21.010057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1588e6-b8b0-407d-9bd6-ff3843d17394 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:21.010355 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1588e6-b8b0-407d-9bd6-ff3843d17394 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:21.010639 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1588e6-b8b0-407d-9bd6-ff3843d17394 None admin] PATH_INFO: `/v3/services/network` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:21.011760 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1588e6-b8b0-407d-9bd6-ff3843d17394 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:21.012713 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1588e6-b8b0-407d-9bd6-ff3843d17394 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:21.013884 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec83777-fddf-456c-8278-67928ab64245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:21.014515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec83777-fddf-456c-8278-67928ab64245 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:21.014799 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec83777-fddf-456c-8278-67928ab64245 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:21.015078 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec83777-fddf-456c-8278-67928ab64245 None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:21.015765 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec83777-fddf-456c-8278-67928ab64245 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:21.016098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec83777-fddf-456c-8278-67928ab64245 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:21.018123 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec83777-fddf-456c-8278-67928ab64245 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:21.024427 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 411/812] 10.222.0.40 () {56 vars in 1124 bytes} [Fri Aug 5 15:13:20 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:21.025255 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec83777-fddf-456c-8278-67928ab64245 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:21.025828 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-0d1588e6-b8b0-407d-9bd6-ff3843d17394 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 15:13:21.025828 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:21.025828 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 15:13:21.025828 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 15:13:21.025828 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 15:13:21.025828 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 15:13:21.025828 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 15:13:21.025828 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:21.025828 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 15:13:21.025828 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:21.025828 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 15:13:21.025828 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:21.025828 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:21.025828 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:21.025828 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:21.025828 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:21.025828 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:21.026780 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:21.026780 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:21.026780 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:21.026780 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:21.026780 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:21.026780 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:21.026780 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 15:13:21.026780 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 15:13:21.026780 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 15:13:21.026780 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 15:13:21.026780 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:21.026780 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:21.026780 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:21.026780 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:21.026780 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:21.026780 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:21.027985 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:21.027985 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:21.027985 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:21.027985 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:21.027985 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:21.027985 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:21.027985 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:21.027985 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:21.027985 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:21.027985 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:21.027985 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 15:13:21.027985 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:21.027985 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 15:13:21.027985 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:21.027985 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 402/813] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:13:20 2022] GET /identity/v3/services/network => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:21.029526 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1588e6-b8b0-407d-9bd6-ff3843d17394 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:21.148484 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1588e6-b8b0-407d-9bd6-ff3843d17394 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:21.152074 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec83777-fddf-456c-8278-67928ab64245 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:21.180566 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:21.189790 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2beab59-5f82-4e6e-83fb-7aa5f283fb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:21.190515 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2beab59-5f82-4e6e-83fb-7aa5f283fb09 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:21.190890 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2beab59-5f82-4e6e-83fb-7aa5f283fb09 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:21.191283 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2beab59-5f82-4e6e-83fb-7aa5f283fb09 None admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:21.192056 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2beab59-5f82-4e6e-83fb-7aa5f283fb09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:21.192443 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2beab59-5f82-4e6e-83fb-7aa5f283fb09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:21.193768 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2beab59-5f82-4e6e-83fb-7aa5f283fb09 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:21.199969 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465daaed-4726-4521-9fb7-e805bef7ff4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:21.199969 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465daaed-4726-4521-9fb7-e805bef7ff4c None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:21.199969 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465daaed-4726-4521-9fb7-e805bef7ff4c None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:21.199969 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465daaed-4726-4521-9fb7-e805bef7ff4c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:21.207685 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465daaed-4726-4521-9fb7-e805bef7ff4c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:21.209312 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465daaed-4726-4521-9fb7-e805bef7ff4c None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:21.210351 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 403/814] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:21.211188 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2beab59-5f82-4e6e-83fb-7aa5f283fb09 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:21.225860 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-465daaed-4726-4521-9fb7-e805bef7ff4c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 15:13:21.225860 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:21.225860 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:21.225860 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:21.225860 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:21.225860 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:21.225860 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:21.225860 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:21.225860 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:21.225860 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:21.225860 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:21.225860 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:21.225860 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 15:13:21.225860 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 15:13:21.225860 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 15:13:21.225860 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 15:13:21.226888 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:21.226888 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:21.226888 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 15:13:21.226888 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:21.226888 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 15:13:21.226888 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:21.226888 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:21.226888 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:21.226888 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:21.226888 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:21.226888 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:21.226888 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:21.226888 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:21.226888 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:21.226888 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:21.226888 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:21.226888 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:21.226888 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:21.227813 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:21.227813 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:21.227813 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 15:13:21.227813 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 15:13:21.227813 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 15:13:21.227813 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 15:13:21.227813 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 15:13:21.227813 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 15:13:21.227813 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 15:13:21.227813 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:21.232095 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 412/815] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 15:13:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:21.233231 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465daaed-4726-4521-9fb7-e805bef7ff4c None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:21.289446 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:21.356429 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2beab59-5f82-4e6e-83fb-7aa5f283fb09 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:21.358905 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465daaed-4726-4521-9fb7-e805bef7ff4c None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:21.398074 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a60b52-3680-4af3-8fe2-50f9c2bc7cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:21.398722 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a60b52-3680-4af3-8fe2-50f9c2bc7cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:21.399486 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b221a-6c8f-4c29-8c15-b40a7bea6e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:21.399774 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a60b52-3680-4af3-8fe2-50f9c2bc7cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:21.400085 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a60b52-3680-4af3-8fe2-50f9c2bc7cb5 None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:21.400201 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b221a-6c8f-4c29-8c15-b40a7bea6e35 None admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:21.400415 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b221a-6c8f-4c29-8c15-b40a7bea6e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:21.400699 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b221a-6c8f-4c29-8c15-b40a7bea6e35 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:21.401411 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a60b52-3680-4af3-8fe2-50f9c2bc7cb5 None admin] RBAC: Adding query filter params (name=network) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:21.401727 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a60b52-3680-4af3-8fe2-50f9c2bc7cb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:21.403130 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a60b52-3680-4af3-8fe2-50f9c2bc7cb5 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:21.404934 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7b221a-6c8f-4c29-8c15-b40a7bea6e35 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:21.406314 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7b221a-6c8f-4c29-8c15-b40a7bea6e35 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:21.413724 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 413/816] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 5 15:13:21 2022] GET /identity/v3/services?name=network => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:21.415562 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a60b52-3680-4af3-8fe2-50f9c2bc7cb5 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:21.463527 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:21.523658 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 404/817] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:13:21 2022] POST /identity/v3/endpoints => generated 346 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:13:21.524862 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b221a-6c8f-4c29-8c15-b40a7bea6e35 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:21.536122 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a60b52-3680-4af3-8fe2-50f9c2bc7cb5 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:21.585324 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceccb6e-e4bd-4943-a7e3-3b0e96e97e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:21.585901 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceccb6e-e4bd-4943-a7e3-3b0e96e97e21 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:21.586211 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceccb6e-e4bd-4943-a7e3-3b0e96e97e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:21.586568 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceccb6e-e4bd-4943-a7e3-3b0e96e97e21 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:21.596448 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceccb6e-e4bd-4943-a7e3-3b0e96e97e21 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:21.598291 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceccb6e-e4bd-4943-a7e3-3b0e96e97e21 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:21.613518 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-7ceccb6e-e4bd-4943-a7e3-3b0e96e97e21 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 15:13:21.613518 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:21.613518 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:21.613518 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:21.613518 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:21.613518 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:21.613518 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:21.613518 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:21.613518 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:21.613518 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:21.613518 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:21.613518 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:21.613518 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 15:13:21.613518 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 15:13:21.613518 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 15:13:21.613518 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 15:13:21.618810 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:21.618810 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:21.618810 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 15:13:21.618810 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:21.618810 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 15:13:21.618810 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:21.618810 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:21.618810 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:21.618810 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:21.618810 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:21.618810 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:21.618810 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:21.618810 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:21.618810 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:21.618810 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:21.618810 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:21.618810 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:21.618810 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:21.619779 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:21.619779 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:21.619779 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 15:13:21.619779 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 15:13:21.619779 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 15:13:21.619779 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 15:13:21.619779 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 15:13:21.619779 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 15:13:21.619779 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 15:13:21.619779 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:21.619779 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 414/818] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Aug 5 15:13:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:21.619779 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceccb6e-e4bd-4943-a7e3-3b0e96e97e21 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:21.671361 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:21.676248 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b221a-6c8f-4c29-8c15-b40a7bea6e35 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:21.728647 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc2da4-5e88-4a57-9257-32eba0d3f45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:21.732235 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc2da4-5e88-4a57-9257-32eba0d3f45a None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:21.732533 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc2da4-5e88-4a57-9257-32eba0d3f45a None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:21.732813 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc2da4-5e88-4a57-9257-32eba0d3f45a None admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:21.733626 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc2da4-5e88-4a57-9257-32eba0d3f45a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:21.733940 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc2da4-5e88-4a57-9257-32eba0d3f45a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:21.736864 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc2da4-5e88-4a57-9257-32eba0d3f45a None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:21.743975 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:21.752383 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceccb6e-e4bd-4943-a7e3-3b0e96e97e21 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:21.758854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 405/819] 10.222.0.40 () {56 vars in 1135 bytes} [Fri Aug 5 15:13:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:21.759489 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc2da4-5e88-4a57-9257-32eba0d3f45a None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:21.801071 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d99c2-bf4f-4a08-9a53-e179927c0ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:21.802063 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d99c2-bf4f-4a08-9a53-e179927c0ba0 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:21.802544 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d99c2-bf4f-4a08-9a53-e179927c0ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:21.803013 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d99c2-bf4f-4a08-9a53-e179927c0ba0 None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:21.803940 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6d99c2-bf4f-4a08-9a53-e179927c0ba0 None admin] RBAC: Adding query filter params (type=network) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:21.804950 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6d99c2-bf4f-4a08-9a53-e179927c0ba0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:21.806885 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6d99c2-bf4f-4a08-9a53-e179927c0ba0 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:21.818462 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 415/820] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 5 15:13:21 2022] GET /identity/v3/services?type=network => generated 350 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:21.818462 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d99c2-bf4f-4a08-9a53-e179927c0ba0 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:21.883663 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc2da4-5e88-4a57-9257-32eba0d3f45a None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:21.925735 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd6de8-b981-441d-bd7d-4a0f0f8af745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:21.926390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd6de8-b981-441d-bd7d-4a0f0f8af745 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:21.926725 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd6de8-b981-441d-bd7d-4a0f0f8af745 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:21.927013 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd6de8-b981-441d-bd7d-4a0f0f8af745 None admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:21.927817 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccd6de8-b981-441d-bd7d-4a0f0f8af745 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:21.928260 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccd6de8-b981-441d-bd7d-4a0f0f8af745 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:21.929786 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccd6de8-b981-441d-bd7d-4a0f0f8af745 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:21.949397 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 406/821] 10.222.0.40 () {56 vars in 1149 bytes} [Fri Aug 5 15:13:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:21.950322 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd6de8-b981-441d-bd7d-4a0f0f8af745 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:21.958930 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d99c2-bf4f-4a08-9a53-e179927c0ba0 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:21.975043 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:21.998181 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750f37b7-a75d-41f7-ae2c-2132ef129bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:21.998824 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f37b7-a75d-41f7-ae2c-2132ef129bbf None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:21.999110 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f37b7-a75d-41f7-ae2c-2132ef129bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:21.999387 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f37b7-a75d-41f7-ae2c-2132ef129bbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:22.011364 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750f37b7-a75d-41f7-ae2c-2132ef129bbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:22.012895 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750f37b7-a75d-41f7-ae2c-2132ef129bbf None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:22.028835 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-750f37b7-a75d-41f7-ae2c-2132ef129bbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:22.028835 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:22.028835 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:22.028835 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:22.028835 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:22.028835 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:22.028835 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:22.028835 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:22.028835 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:22.028835 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:22.028835 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:22.028835 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:22.028835 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:22.028835 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:22.028835 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:22.029781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:22.029781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:22.029781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:22.029781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:22.029781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:22.029781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:22.029781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:22.029781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:22.029781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:22.029781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:22.029781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:22.029781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:22.029781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:22.029781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:22.029781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:22.029781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:22.029781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:22.031077 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:22.031077 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:22.031077 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:22.031077 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:22.031077 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 416/822] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:22.031365 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750f37b7-a75d-41f7-ae2c-2132ef129bbf None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:22.048476 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:22.066241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd6de8-b981-441d-bd7d-4a0f0f8af745 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:22.114497 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f72b42-e93e-4658-8c74-85bc45405940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:22.115163 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f72b42-e93e-4658-8c74-85bc45405940 None admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:22.115462 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f72b42-e93e-4658-8c74-85bc45405940 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:22.115746 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f72b42-e93e-4658-8c74-85bc45405940 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:22.116578 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f72b42-e93e-4658-8c74-85bc45405940 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:22.118065 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f72b42-e93e-4658-8c74-85bc45405940 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:22.150784 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750f37b7-a75d-41f7-ae2c-2132ef129bbf None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:22.186516 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00c3678-12da-4908-b4e7-23aa731267d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:22.187155 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c3678-12da-4908-b4e7-23aa731267d7 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:22.187441 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c3678-12da-4908-b4e7-23aa731267d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:22.187719 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c3678-12da-4908-b4e7-23aa731267d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:22.194019 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 407/823] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:13:21 2022] POST /identity/v3/endpoints => generated 337 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:13:22.195605 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f72b42-e93e-4658-8c74-85bc45405940 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:22.214564 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00c3678-12da-4908-b4e7-23aa731267d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:22.216096 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00c3678-12da-4908-b4e7-23aa731267d7 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:22.242515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-d00c3678-12da-4908-b4e7-23aa731267d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:22.242515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:22.242515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:22.242515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:22.242515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:22.242515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:22.242515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:22.242515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:22.242515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:22.242515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:22.242515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:22.242515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:22.242515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:22.242515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:22.242515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:22.243492 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:22.243492 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:22.243492 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:22.243492 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:22.243492 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:22.243492 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:22.243492 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:22.243492 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:22.243492 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:22.243492 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:22.243492 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:22.243492 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:22.243492 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:22.243492 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:22.243492 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:22.243492 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:22.243492 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:22.244442 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:22.244442 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:22.244442 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:22.244442 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:22.248511 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 417/824] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:22.253084 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00c3678-12da-4908-b4e7-23aa731267d7 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:22.302169 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:22.331764 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f72b42-e93e-4658-8c74-85bc45405940 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:22.405321 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:22.408223 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfca15a-f178-4f51-8073-cb97751b110d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:22.408994 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfca15a-f178-4f51-8073-cb97751b110d None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:22.409348 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfca15a-f178-4f51-8073-cb97751b110d None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:22.409695 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfca15a-f178-4f51-8073-cb97751b110d None admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:22.410466 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfca15a-f178-4f51-8073-cb97751b110d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:22.410862 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfca15a-f178-4f51-8073-cb97751b110d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:22.412349 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfca15a-f178-4f51-8073-cb97751b110d None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:22.425896 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 408/825] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:22.430683 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfca15a-f178-4f51-8073-cb97751b110d None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:22.452169 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00c3678-12da-4908-b4e7-23aa731267d7 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:22.489372 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced653d0-fc38-446c-b95e-5ad4b27c3991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:22.490384 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced653d0-fc38-446c-b95e-5ad4b27c3991 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:22.490811 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced653d0-fc38-446c-b95e-5ad4b27c3991 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:22.491227 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced653d0-fc38-446c-b95e-5ad4b27c3991 None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:22.492100 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced653d0-fc38-446c-b95e-5ad4b27c3991 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:22.492713 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced653d0-fc38-446c-b95e-5ad4b27c3991 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:22.494397 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced653d0-fc38-446c-b95e-5ad4b27c3991 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:22.508641 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 418/826] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:22.513722 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced653d0-fc38-446c-b95e-5ad4b27c3991 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:22.538131 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfca15a-f178-4f51-8073-cb97751b110d None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:22.567906 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:22.577490 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec7ee51-d85f-4008-90aa-6c2464a7dd26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:22.578305 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7ee51-d85f-4008-90aa-6c2464a7dd26 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:22.578415 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7ee51-d85f-4008-90aa-6c2464a7dd26 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:22.578692 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7ee51-d85f-4008-90aa-6c2464a7dd26 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:22.586647 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec7ee51-d85f-4008-90aa-6c2464a7dd26 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:22.588249 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec7ee51-d85f-4008-90aa-6c2464a7dd26 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:22.600057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-aec7ee51-d85f-4008-90aa-6c2464a7dd26 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:22.600057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:22.600057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:22.600057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:22.600057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:22.600057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:22.600057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:22.600057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:22.600057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:22.600057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:22.600057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:22.600057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:22.600057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 15:13:22.600057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 15:13:22.600057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 15:13:22.600995 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 15:13:22.600995 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:22.600995 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:22.600995 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:22.600995 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:22.600995 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:22.600995 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:22.600995 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:22.600995 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:22.600995 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:22.600995 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:22.600995 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:22.600995 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:22.600995 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:22.600995 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:22.600995 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:22.600995 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:22.602266 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 15:13:22.602266 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 15:13:22.602266 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 15:13:22.602266 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 15:13:22.602266 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 15:13:22.602266 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 15:13:22.602266 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:22.602266 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:22.602266 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 409/827] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:13:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:13:22.607265 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec7ee51-d85f-4008-90aa-6c2464a7dd26 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:22.631686 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced653d0-fc38-446c-b95e-5ad4b27c3991 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:22.657617 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:22.669594 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325b3532-11a0-495f-b12b-9564ecc2aada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:22.671565 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b3532-11a0-495f-b12b-9564ecc2aada None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:22.671565 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b3532-11a0-495f-b12b-9564ecc2aada None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:22.671565 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b3532-11a0-495f-b12b-9564ecc2aada None admin] PATH_INFO: `/v3/projects/service` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:22.678823 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325b3532-11a0-495f-b12b-9564ecc2aada None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:22.680525 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325b3532-11a0-495f-b12b-9564ecc2aada None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:22.692776 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-325b3532-11a0-495f-b12b-9564ecc2aada None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:22.692776 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:22.692776 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:22.692776 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:22.692776 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:22.692776 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:22.692776 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:22.692776 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:22.692776 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:22.692776 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:22.692776 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:22.692776 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:22.692776 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 15:13:22.692776 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 15:13:22.692776 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 15:13:22.693696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 15:13:22.693696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:22.693696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:22.693696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:22.693696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:22.693696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:22.693696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:22.693696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:22.693696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:22.693696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:22.693696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:22.693696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:22.693696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:22.693696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:22.693696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:22.693696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:22.693696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:22.694556 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 15:13:22.694556 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 15:13:22.694556 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 15:13:22.694556 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 15:13:22.694556 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 15:13:22.694556 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 15:13:22.694556 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:22.694556 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:22.701433 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 419/828] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:13:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:13:22.702932 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325b3532-11a0-495f-b12b-9564ecc2aada None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:22.716685 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec7ee51-d85f-4008-90aa-6c2464a7dd26 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:22.754828 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ebe9f6-9882-4f1d-b336-226dc849151a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:22.755507 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ebe9f6-9882-4f1d-b336-226dc849151a None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:22.755803 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ebe9f6-9882-4f1d-b336-226dc849151a None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:22.756090 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ebe9f6-9882-4f1d-b336-226dc849151a None admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:22.757007 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ebe9f6-9882-4f1d-b336-226dc849151a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:22.757320 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ebe9f6-9882-4f1d-b336-226dc849151a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:22.758800 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ebe9f6-9882-4f1d-b336-226dc849151a None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:22.773757 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 410/829] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 15:13:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:22.779507 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ebe9f6-9882-4f1d-b336-226dc849151a None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:22.810067 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325b3532-11a0-495f-b12b-9564ecc2aada None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:22.834962 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:22.846736 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c6c551-146e-40a1-ad0d-46d2ee0b0b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:22.847359 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6c551-146e-40a1-ad0d-46d2ee0b0b15 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:22.847642 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6c551-146e-40a1-ad0d-46d2ee0b0b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:22.847910 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6c551-146e-40a1-ad0d-46d2ee0b0b15 None admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:22.848709 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c6c551-146e-40a1-ad0d-46d2ee0b0b15 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:22.849023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c6c551-146e-40a1-ad0d-46d2ee0b0b15 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:22.850459 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c6c551-146e-40a1-ad0d-46d2ee0b0b15 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:22.868202 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 420/830] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 15:13:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:22.869872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c6c551-146e-40a1-ad0d-46d2ee0b0b15 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:22.886859 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ebe9f6-9882-4f1d-b336-226dc849151a None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:22.923475 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e353842d-9915-432e-af3c-9aad702e7318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:22.924522 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353842d-9915-432e-af3c-9aad702e7318 None admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:22.925000 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353842d-9915-432e-af3c-9aad702e7318 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:22.925456 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353842d-9915-432e-af3c-9aad702e7318 None admin] PATH_INFO: `/v3/projects/06f27ab8886a4c0d81c4d4e3fa100a83/users/85154ce7446245738edcc8702dbf6861/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:22.926639 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:22.929327 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e353842d-9915-432e-af3c-9aad702e7318 None admin] RBAC: Authorizing `identity:create_grant(project_id=06f27ab8886a4c0d81c4d4e3fa100a83, user_id=85154ce7446245738edcc8702dbf6861, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:22.930941 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e353842d-9915-432e-af3c-9aad702e7318 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:22.981957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c6c551-146e-40a1-ad0d-46d2ee0b0b15 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:23.016366 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 411/831] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 15:13:22 2022] PUT /identity/v3/projects/06f27ab8886a4c0d81c4d4e3fa100a83/users/85154ce7446245738edcc8702dbf6861/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:13:23.023857 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda01315-0ea4-47a4-a2ab-7b27236177be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:23.024996 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda01315-0ea4-47a4-a2ab-7b27236177be None admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:23.025492 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda01315-0ea4-47a4-a2ab-7b27236177be None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:23.025948 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda01315-0ea4-47a4-a2ab-7b27236177be None admin] PATH_INFO: `/v3/projects/06f27ab8886a4c0d81c4d4e3fa100a83/users/1654fb6a81e14d6b99ab495bb6d029d7/roles/266a023f66e043f99efe2941fe58d481` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:23.029421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a11e04-afad-4235-99e1-cf9dfac326ee None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:23.029856 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a11e04-afad-4235-99e1-cf9dfac326ee None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:23.030073 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a11e04-afad-4235-99e1-cf9dfac326ee None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:23.031689 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 412/832] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:23.037454 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd6d1e-9d4e-40d4-94ca-cfb76370eb86 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:23.037744 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd6d1e-9d4e-40d4-94ca-cfb76370eb86 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:23.038028 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd6d1e-9d4e-40d4-94ca-cfb76370eb86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:23.044307 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda01315-0ea4-47a4-a2ab-7b27236177be None admin] RBAC: Authorizing `identity:create_grant(project_id=06f27ab8886a4c0d81c4d4e3fa100a83, user_id=1654fb6a81e14d6b99ab495bb6d029d7, role_id=266a023f66e043f99efe2941fe58d481)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:23.045890 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda01315-0ea4-47a4-a2ab-7b27236177be None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:23.068149 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-9cbd6d1e-9d4e-40d4-94ca-cfb76370eb86 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:23.069396 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-9cbd6d1e-9d4e-40d4-94ca-cfb76370eb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:23.122173 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 421/833] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 15:13:22 2022] PUT /identity/v3/projects/06f27ab8886a4c0d81c4d4e3fa100a83/users/1654fb6a81e14d6b99ab495bb6d029d7/roles/266a023f66e043f99efe2941fe58d481 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:13:23.159575 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 413/834] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:23 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:13:23.186616 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda01315-0ea4-47a4-a2ab-7b27236177be None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:23.293791 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:23.303915 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda01315-0ea4-47a4-a2ab-7b27236177be None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:23.339203 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c85d50-6dc7-4a07-96e3-c60eaad2b348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:23.339855 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c85d50-6dc7-4a07-96e3-c60eaad2b348 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:23.340167 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c85d50-6dc7-4a07-96e3-c60eaad2b348 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:23.340446 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c85d50-6dc7-4a07-96e3-c60eaad2b348 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:23.341074 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c85d50-6dc7-4a07-96e3-c60eaad2b348 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:23.342441 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c85d50-6dc7-4a07-96e3-c60eaad2b348 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:23.352761 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-97c85d50-6dc7-4a07-96e3-c60eaad2b348 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 15:13:23.352761 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:23.352761 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 15:13:23.352761 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 15:13:23.352761 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 15:13:23.352761 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 15:13:23.352761 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 15:13:23.352761 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:23.352761 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 15:13:23.352761 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:23.352761 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 15:13:23.352761 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:23.352761 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:23.352761 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:23.352761 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:23.352761 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:23.353689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:23.353689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:23.353689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:23.353689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:23.353689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:23.353689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:23.353689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:23.353689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 15:13:23.353689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 15:13:23.353689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 15:13:23.353689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 15:13:23.353689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:23.353689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:23.353689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:23.353689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:23.353689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:23.355118 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:23.355118 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:23.355118 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:23.355118 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:23.355118 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:23.355118 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:23.355118 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:23.355118 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:23.355118 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:23.355118 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:23.355118 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:23.355118 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 15:13:23.355118 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:23.355118 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 15:13:23.355118 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:23.355118 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 422/835] 10.222.0.40 () {56 vars in 1132 bytes} [Fri Aug 5 15:13:23 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:23.359714 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd6d1e-9d4e-40d4-94ca-cfb76370eb86 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:23.403775 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:23.467805 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd6d1e-9d4e-40d4-94ca-cfb76370eb86 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:23.503994 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce2cf7d-4b10-4663-adf9-9d36b63b5454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:23.504654 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2cf7d-4b10-4663-adf9-9d36b63b5454 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:23.504942 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2cf7d-4b10-4663-adf9-9d36b63b5454 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:23.505228 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2cf7d-4b10-4663-adf9-9d36b63b5454 None admin] PATH_INFO: `/v3/services` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:23.505977 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce2cf7d-4b10-4663-adf9-9d36b63b5454 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:23.506295 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce2cf7d-4b10-4663-adf9-9d36b63b5454 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:23.507669 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce2cf7d-4b10-4663-adf9-9d36b63b5454 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:23.512857 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 414/836] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 5 15:13:23 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:23.518557 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c85d50-6dc7-4a07-96e3-c60eaad2b348 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:23.628768 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c85d50-6dc7-4a07-96e3-c60eaad2b348 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:23.668098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23b1acb-c61d-4ace-81b3-4aa72fcf0e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:23.669091 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b1acb-c61d-4ace-81b3-4aa72fcf0e52 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:23.669588 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b1acb-c61d-4ace-81b3-4aa72fcf0e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:23.670067 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b1acb-c61d-4ace-81b3-4aa72fcf0e52 None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:23.670934 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23b1acb-c61d-4ace-81b3-4aa72fcf0e52 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:23.671432 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23b1acb-c61d-4ace-81b3-4aa72fcf0e52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:23.673053 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23b1acb-c61d-4ace-81b3-4aa72fcf0e52 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:23.678447 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 423/837] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 5 15:13:23 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:23.755060 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4131a0a-ef6b-4f13-b609-7289bdde5d03 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:23.755364 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4131a0a-ef6b-4f13-b609-7289bdde5d03 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:23.755552 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4131a0a-ef6b-4f13-b609-7289bdde5d03 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:23.757222 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 415/838] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:23.762656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea2bf7-1b59-4163-b92d-895d7e274bc1 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:23.763148 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea2bf7-1b59-4163-b92d-895d7e274bc1 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:23.763617 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea2bf7-1b59-4163-b92d-895d7e274bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:23.784711 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-bdea2bf7-1b59-4163-b92d-895d7e274bc1 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:23.786264 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-bdea2bf7-1b59-4163-b92d-895d7e274bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:23.806611 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:23.863135 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 424/839] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:23 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:13:23.885056 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4131a0a-ef6b-4f13-b609-7289bdde5d03 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:23.989230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4131a0a-ef6b-4f13-b609-7289bdde5d03 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:23.990771 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:24.026127 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6157a0ed-fcfb-485a-b207-5c2e8e04a865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:24.026774 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157a0ed-fcfb-485a-b207-5c2e8e04a865 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:24.027062 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157a0ed-fcfb-485a-b207-5c2e8e04a865 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:24.027345 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157a0ed-fcfb-485a-b207-5c2e8e04a865 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:24.027978 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6157a0ed-fcfb-485a-b207-5c2e8e04a865 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:24.029392 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6157a0ed-fcfb-485a-b207-5c2e8e04a865 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:24.038656 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-6157a0ed-fcfb-485a-b207-5c2e8e04a865 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 15:13:24.038656 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:24.038656 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 15:13:24.038656 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 15:13:24.038656 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 15:13:24.038656 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 15:13:24.038656 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 15:13:24.038656 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:24.038656 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 15:13:24.038656 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:24.038656 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 15:13:24.038656 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:24.038656 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:24.038656 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:24.038656 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:24.038656 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:24.038656 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:24.039631 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:24.039631 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:24.039631 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:24.039631 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:24.039631 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:24.039631 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:24.039631 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 15:13:24.039631 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 15:13:24.039631 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 15:13:24.039631 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 15:13:24.039631 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:24.039631 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:24.039631 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:24.039631 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:24.039631 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:24.039631 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:24.040498 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:24.040498 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:24.040498 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:24.040498 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:24.040498 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:24.040498 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:24.040498 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:24.040498 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:24.040498 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:24.040498 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:24.040498 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 15:13:24.040498 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:24.040498 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 15:13:24.040498 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:24.040498 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 416/840] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 15:13:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:24.047539 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdea2bf7-1b59-4163-b92d-895d7e274bc1 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:24.163479 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdea2bf7-1b59-4163-b92d-895d7e274bc1 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:24.200432 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a257d45-453f-4f4f-a820-6d62d80e4bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:24.201060 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a257d45-453f-4f4f-a820-6d62d80e4bae None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:24.201346 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a257d45-453f-4f4f-a820-6d62d80e4bae None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:24.201626 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a257d45-453f-4f4f-a820-6d62d80e4bae None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:24.202311 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a257d45-453f-4f4f-a820-6d62d80e4bae None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:24.202621 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a257d45-453f-4f4f-a820-6d62d80e4bae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:24.204590 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a257d45-453f-4f4f-a820-6d62d80e4bae None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:24.209686 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 425/841] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 15:13:24 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:24.217352 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6157a0ed-fcfb-485a-b207-5c2e8e04a865 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:24.325835 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6157a0ed-fcfb-485a-b207-5c2e8e04a865 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:24.362928 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629110a3-409c-4710-b16e-175cb3e71203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:24.363581 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629110a3-409c-4710-b16e-175cb3e71203 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:24.363881 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629110a3-409c-4710-b16e-175cb3e71203 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:24.364186 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629110a3-409c-4710-b16e-175cb3e71203 None admin] PATH_INFO: `/v3/services` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:24.364913 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629110a3-409c-4710-b16e-175cb3e71203 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:24.365228 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629110a3-409c-4710-b16e-175cb3e71203 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:24.366583 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629110a3-409c-4710-b16e-175cb3e71203 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:24.371473 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 417/842] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 15:13:24 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:25.229894 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4dda9e-598d-4e01-8f35-601d368facfd None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:25.230416 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4dda9e-598d-4e01-8f35-601d368facfd None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:25.230691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4dda9e-598d-4e01-8f35-601d368facfd None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:25.232429 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 426/843] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:25.238404 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece08163-1c5f-410a-bd0a-e66d43057c2b None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:25.238700 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece08163-1c5f-410a-bd0a-e66d43057c2b None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:25.238981 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece08163-1c5f-410a-bd0a-e66d43057c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:25.257282 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beaece8-bcb4-41ad-8865-c1f960b4dd4e None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:25.257664 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beaece8-bcb4-41ad-8865-c1f960b4dd4e None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:25.257957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beaece8-bcb4-41ad-8865-c1f960b4dd4e None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:25.258509 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-ece08163-1c5f-410a-bd0a-e66d43057c2b None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:25.259564 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 427/844] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 15:13:25.259742 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-ece08163-1c5f-410a-bd0a-e66d43057c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:25.265767 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83f7d0-b75d-41d0-be2b-9c84b8d2a3a1 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:25.266074 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83f7d0-b75d-41d0-be2b-9c84b8d2a3a1 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:25.266353 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83f7d0-b75d-41d0-be2b-9c84b8d2a3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:25.290664 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-be83f7d0-b75d-41d0-be2b-9c84b8d2a3a1 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:25.291620 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:25.292924 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-be83f7d0-b75d-41d0-be2b-9c84b8d2a3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:25.304863 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:25.337740 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 418/845] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:25 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:13:25.362124 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece08163-1c5f-410a-bd0a-e66d43057c2b None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:25.365107 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 428/846] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:25 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 15:13:25.387974 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be83f7d0-b75d-41d0-be2b-9c84b8d2a3a1 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:25.475121 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece08163-1c5f-410a-bd0a-e66d43057c2b None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:25.519071 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be83f7d0-b75d-41d0-be2b-9c84b8d2a3a1 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:25.523265 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f98dc5-2f8a-4c85-8125-35fada3b2d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:25.524227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f98dc5-2f8a-4c85-8125-35fada3b2d87 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:25.524690 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f98dc5-2f8a-4c85-8125-35fada3b2d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:25.525169 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f98dc5-2f8a-4c85-8125-35fada3b2d87 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:25.539896 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f98dc5-2f8a-4c85-8125-35fada3b2d87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:13:25.545209 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-68f98dc5-2f8a-4c85-8125-35fada3b2d87 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:13:25.545209 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:25.545209 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:25.545209 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:25.545209 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:25.545209 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:25.545209 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:25.545209 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:25.545209 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:25.545209 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:25.545209 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:25.545209 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:25.545209 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:13:25.545209 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:13:25.545209 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:13:25.545209 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:13:25.551217 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:13:25.551217 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:13:25.551217 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:13:25.551217 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:13:25.551217 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:25.551217 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:25.551217 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:25.551217 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:25.551217 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:25.551217 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:25.551217 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:25.551217 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:25.551217 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:25.551217 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:25.551217 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:25.551217 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:25.552089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:25.552089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:25.552089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:25.552089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:25.552089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:13:25.552089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:13:25.552089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:13:25.552089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:13:25.552089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:13:25.552089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:13:25.552089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:13:25.552089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:25.552089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 419/847] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 15:13:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:13:25.553408 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f98dc5-2f8a-4c85-8125-35fada3b2d87 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:25.567206 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ea620-22a8-44b6-8549-83379ab377cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:25.568203 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ea620-22a8-44b6-8549-83379ab377cf None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:25.568707 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ea620-22a8-44b6-8549-83379ab377cf None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:25.569182 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ea620-22a8-44b6-8549-83379ab377cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:25.580707 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ea620-22a8-44b6-8549-83379ab377cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:13:25.582272 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:25.586400 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-e58ea620-22a8-44b6-8549-83379ab377cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:13:25.586400 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:25.586400 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:25.586400 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:25.586400 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:25.586400 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:25.586400 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:25.586400 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:25.586400 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:25.586400 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:25.586400 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:25.586400 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:25.586400 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:13:25.586400 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:13:25.586400 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:13:25.586400 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:13:25.587358 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:13:25.587358 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:13:25.587358 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:13:25.587358 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:13:25.587358 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:25.587358 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:25.587358 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:25.587358 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:25.587358 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:25.587358 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:25.587358 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:25.587358 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:25.587358 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:25.587358 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:25.587358 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:25.587358 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:25.588238 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:25.588238 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:25.588238 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:25.588238 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:25.588238 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:13:25.588238 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:13:25.588238 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:13:25.588238 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:13:25.588238 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:13:25.588238 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:13:25.588238 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:13:25.588238 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:25.590559 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 429/848] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 15:13:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:13:25.600096 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ea620-22a8-44b6-8549-83379ab377cf None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:25.606261 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:25.676430 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f98dc5-2f8a-4c85-8125-35fada3b2d87 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:25.718289 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ea620-22a8-44b6-8549-83379ab377cf None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:25.723294 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce8eea-8694-427b-ba6c-bdb4f814eb3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:25.724039 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce8eea-8694-427b-ba6c-bdb4f814eb3a None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:25.724446 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce8eea-8694-427b-ba6c-bdb4f814eb3a None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:25.724821 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce8eea-8694-427b-ba6c-bdb4f814eb3a None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:25.725686 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccce8eea-8694-427b-ba6c-bdb4f814eb3a None admin] RBAC: Adding query filter params (name=service) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:25.726038 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccce8eea-8694-427b-ba6c-bdb4f814eb3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:25.727821 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccce8eea-8694-427b-ba6c-bdb4f814eb3a None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:25.744093 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 420/849] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 15:13:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:25.745244 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce8eea-8694-427b-ba6c-bdb4f814eb3a None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:25.758592 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f659d-61a3-4980-bda9-4e46306c1ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:25.759213 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f659d-61a3-4980-bda9-4e46306c1ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:25.759495 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f659d-61a3-4980-bda9-4e46306c1ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:25.759771 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f659d-61a3-4980-bda9-4e46306c1ea5 None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:25.760532 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1f659d-61a3-4980-bda9-4e46306c1ea5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:25.760836 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1f659d-61a3-4980-bda9-4e46306c1ea5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:25.762401 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1f659d-61a3-4980-bda9-4e46306c1ea5 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:25.777044 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 430/850] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:13:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:25.778341 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f659d-61a3-4980-bda9-4e46306c1ea5 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:25.849795 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:25.854503 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:25.867520 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce8eea-8694-427b-ba6c-bdb4f814eb3a None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:25.894307 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f659d-61a3-4980-bda9-4e46306c1ea5 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:25.905235 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6274d160-5a20-497a-8640-6714958cf75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:25.906195 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274d160-5a20-497a-8640-6714958cf75d None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:25.906489 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274d160-5a20-497a-8640-6714958cf75d None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:25.907327 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274d160-5a20-497a-8640-6714958cf75d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:25.918496 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6274d160-5a20-497a-8640-6714958cf75d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:25.919947 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6274d160-5a20-497a-8640-6714958cf75d None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:25.931540 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60521059-3484-41d6-a7e1-f8086ab1d319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:25.932391 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60521059-3484-41d6-a7e1-f8086ab1d319 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:25.932842 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60521059-3484-41d6-a7e1-f8086ab1d319 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:25.933270 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60521059-3484-41d6-a7e1-f8086ab1d319 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:25.934681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-6274d160-5a20-497a-8640-6714958cf75d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:25.934681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:25.934681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:25.934681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:25.934681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:25.934681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:25.934681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:25.934681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:25.934681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:25.934681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:25.934681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:25.934681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:25.934681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:25.934681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:25.934681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:25.935735 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:25.935735 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:25.935735 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:25.935735 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:25.935735 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:25.935735 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:25.935735 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:25.935735 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:25.935735 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:25.935735 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:25.935735 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:25.935735 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:25.935735 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:25.935735 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:25.935735 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:25.935735 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:25.935735 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:25.936663 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:25.936663 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:25.936663 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:25.936663 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:25.936663 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 421/851] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:25.937045 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bcc0a9-db62-4d0a-bd71-a6ee8f3ad8dd None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:25.937294 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bcc0a9-db62-4d0a-bd71-a6ee8f3ad8dd None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:25.937575 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bcc0a9-db62-4d0a-bd71-a6ee8f3ad8dd None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:25.939065 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 422/852] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:25.943302 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bcc0a9-db62-4d0a-bd71-a6ee8f3ad8dd None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:25.951708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60521059-3484-41d6-a7e1-f8086ab1d319 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:25.953214 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60521059-3484-41d6-a7e1-f8086ab1d319 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:25.967847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-60521059-3484-41d6-a7e1-f8086ab1d319 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:25.967847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:25.967847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:25.967847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:25.967847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:25.967847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:25.967847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:25.967847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:25.967847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:25.967847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:25.967847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:25.967847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:25.967847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:25.967847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:25.967847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:25.968792 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:25.968792 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:25.968792 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:25.968792 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:25.968792 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:25.968792 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:25.968792 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:25.968792 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:25.968792 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:25.968792 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:25.968792 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:25.968792 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:25.968792 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:25.968792 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:25.968792 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:25.968792 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:25.968792 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:25.970348 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:25.970348 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:25.970348 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:25.970348 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:25.972382 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 431/853] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:13:25.973881 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed5d61-7d44-4dd1-8287-83b91952b0ff None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:25.974411 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed5d61-7d44-4dd1-8287-83b91952b0ff None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:25.974885 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed5d61-7d44-4dd1-8287-83b91952b0ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:25.992745 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-cbed5d61-7d44-4dd1-8287-83b91952b0ff None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:25.993875 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-cbed5d61-7d44-4dd1-8287-83b91952b0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:26.051477 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bcc0a9-db62-4d0a-bd71-a6ee8f3ad8dd None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:26.068510 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 432/854] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:25 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:13:26.069593 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed5d61-7d44-4dd1-8287-83b91952b0ff None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:26.095204 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8976c-b0e3-4735-a383-a8c5283b5b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:26.096130 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8976c-b0e3-4735-a383-a8c5283b5b5b None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:26.096561 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8976c-b0e3-4735-a383-a8c5283b5b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:26.097512 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8976c-b0e3-4735-a383-a8c5283b5b5b None admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:26.098631 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d8976c-b0e3-4735-a383-a8c5283b5b5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:26.099089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d8976c-b0e3-4735-a383-a8c5283b5b5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:26.101575 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d8976c-b0e3-4735-a383-a8c5283b5b5b None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:26.104189 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:26.120401 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 423/855] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:26.121814 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8976c-b0e3-4735-a383-a8c5283b5b5b None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:26.137703 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:26.192830 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed5d61-7d44-4dd1-8287-83b91952b0ff None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:26.228893 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9454cdf-4f70-425d-890c-736283fb324d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:26.229515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9454cdf-4f70-425d-890c-736283fb324d None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:26.229802 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9454cdf-4f70-425d-890c-736283fb324d None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:26.230086 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9454cdf-4f70-425d-890c-736283fb324d None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:26.230785 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9454cdf-4f70-425d-890c-736283fb324d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:26.231102 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9454cdf-4f70-425d-890c-736283fb324d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:26.232524 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9454cdf-4f70-425d-890c-736283fb324d None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:26.241242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8976c-b0e3-4735-a383-a8c5283b5b5b None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:26.248173 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 433/856] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:26.248173 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9454cdf-4f70-425d-890c-736283fb324d None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:26.282832 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ca77a6-9be5-4fe5-91aa-f2f73020ec3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:26.283452 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca77a6-9be5-4fe5-91aa-f2f73020ec3e None admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:26.283733 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca77a6-9be5-4fe5-91aa-f2f73020ec3e None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:26.284005 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca77a6-9be5-4fe5-91aa-f2f73020ec3e None admin] PATH_INFO: `/v3/services` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:26.284726 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ca77a6-9be5-4fe5-91aa-f2f73020ec3e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:26.286087 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ca77a6-9be5-4fe5-91aa-f2f73020ec3e None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:26.345876 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9454cdf-4f70-425d-890c-736283fb324d None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:26.355990 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:26.368561 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 424/857] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 5 15:13:26 2022] POST /identity/v3/services => generated 269 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:13:26.369416 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ca77a6-9be5-4fe5-91aa-f2f73020ec3e None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:26.397360 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c7d21a-248c-4a2d-94b4-89c441b47019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:26.398436 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7d21a-248c-4a2d-94b4-89c441b47019 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:26.399065 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7d21a-248c-4a2d-94b4-89c441b47019 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:26.400497 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7d21a-248c-4a2d-94b4-89c441b47019 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:26.413310 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c7d21a-248c-4a2d-94b4-89c441b47019 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:26.415633 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c7d21a-248c-4a2d-94b4-89c441b47019 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:26.432056 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-56c7d21a-248c-4a2d-94b4-89c441b47019 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 15:13:26.432056 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:26.432056 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:26.432056 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:26.432056 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:26.432056 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:26.432056 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:26.432056 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:26.432056 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:26.432056 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:26.432056 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:26.432056 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:26.432056 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 15:13:26.432056 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 15:13:26.432056 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 15:13:26.432056 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 15:13:26.433130 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:26.433130 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:26.433130 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 15:13:26.433130 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:26.433130 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 15:13:26.433130 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:26.433130 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:26.433130 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:26.433130 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:26.433130 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:26.433130 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:26.433130 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:26.433130 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:26.433130 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:26.433130 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:26.433130 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:26.433130 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:26.433130 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:26.434069 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:26.434069 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:26.434069 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 15:13:26.434069 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 15:13:26.434069 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 15:13:26.434069 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 15:13:26.434069 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 15:13:26.434069 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 15:13:26.434069 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 15:13:26.434069 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:26.436459 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 434/858] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Aug 5 15:13:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:26.440381 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1babbb3-1584-4942-82f8-0c9fb602bcf3 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:26.440668 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1babbb3-1584-4942-82f8-0c9fb602bcf3 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:26.440924 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1babbb3-1584-4942-82f8-0c9fb602bcf3 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:26.442344 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 435/859] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:26.443178 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1babbb3-1584-4942-82f8-0c9fb602bcf3 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:26.450637 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:26.503046 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ca77a6-9be5-4fe5-91aa-f2f73020ec3e None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:26.546549 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a034f94-3a82-4fd9-94ca-9d9e8411e831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:26.547735 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a034f94-3a82-4fd9-94ca-9d9e8411e831 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:26.548020 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a034f94-3a82-4fd9-94ca-9d9e8411e831 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:26.548323 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a034f94-3a82-4fd9-94ca-9d9e8411e831 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:26.556150 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a034f94-3a82-4fd9-94ca-9d9e8411e831 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:26.556544 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1babbb3-1584-4942-82f8-0c9fb602bcf3 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:26.558054 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a034f94-3a82-4fd9-94ca-9d9e8411e831 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:26.572459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-5a034f94-3a82-4fd9-94ca-9d9e8411e831 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 15:13:26.572459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:26.572459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:26.572459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:26.572459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:26.572459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:26.572459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:26.572459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:26.572459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:26.572459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:26.572459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:26.572459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:26.572459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 15:13:26.572459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 15:13:26.572459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 15:13:26.572459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 15:13:26.573423 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:26.573423 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:26.573423 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 15:13:26.573423 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:26.573423 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 15:13:26.573423 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:26.573423 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:26.573423 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:26.573423 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:26.573423 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:26.573423 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:26.573423 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:26.573423 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:26.573423 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:26.573423 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:26.573423 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:26.573423 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:26.573423 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:26.574368 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:26.574368 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:26.574368 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 15:13:26.574368 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 15:13:26.574368 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 15:13:26.574368 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 15:13:26.574368 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 15:13:26.574368 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 15:13:26.574368 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 15:13:26.574368 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:26.574368 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 425/860] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 15:13:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:26.574954 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b94dd-71ce-444f-8cef-99a4de0a8fb5 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:26.575213 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b94dd-71ce-444f-8cef-99a4de0a8fb5 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:26.575317 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b94dd-71ce-444f-8cef-99a4de0a8fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:26.593145 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-590b94dd-71ce-444f-8cef-99a4de0a8fb5 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:26.594388 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-590b94dd-71ce-444f-8cef-99a4de0a8fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:26.598517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af30397d-cf19-4b23-8949-29cc429151fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:26.599324 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30397d-cf19-4b23-8949-29cc429151fe None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:26.599748 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30397d-cf19-4b23-8949-29cc429151fe None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:26.600207 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30397d-cf19-4b23-8949-29cc429151fe None admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:26.601140 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af30397d-cf19-4b23-8949-29cc429151fe None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:26.601590 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af30397d-cf19-4b23-8949-29cc429151fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:26.603180 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af30397d-cf19-4b23-8949-29cc429151fe None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:26.622024 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 436/861] 10.222.0.40 () {56 vars in 1149 bytes} [Fri Aug 5 15:13:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:26.622923 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af30397d-cf19-4b23-8949-29cc429151fe None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:26.653594 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:26.674547 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 426/862] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:26 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:13:26.675789 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b94dd-71ce-444f-8cef-99a4de0a8fb5 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:26.731435 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:26.740226 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af30397d-cf19-4b23-8949-29cc429151fe None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:26.776993 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5886cab5-48e8-4307-aa42-b8daf64c19f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:26.777618 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5886cab5-48e8-4307-aa42-b8daf64c19f1 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:26.777908 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5886cab5-48e8-4307-aa42-b8daf64c19f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:26.778188 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5886cab5-48e8-4307-aa42-b8daf64c19f1 None admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:26.778984 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5886cab5-48e8-4307-aa42-b8daf64c19f1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:26.779299 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5886cab5-48e8-4307-aa42-b8daf64c19f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:26.780761 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5886cab5-48e8-4307-aa42-b8daf64c19f1 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:26.799762 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b94dd-71ce-444f-8cef-99a4de0a8fb5 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:26.807089 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 437/863] 10.222.0.40 () {56 vars in 1135 bytes} [Fri Aug 5 15:13:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:26.807089 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5886cab5-48e8-4307-aa42-b8daf64c19f1 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:26.843580 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe37dc1-8f81-4fac-9126-c777b1b7955e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:26.844373 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe37dc1-8f81-4fac-9126-c777b1b7955e None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:26.844663 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe37dc1-8f81-4fac-9126-c777b1b7955e None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:26.844944 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe37dc1-8f81-4fac-9126-c777b1b7955e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:26.892447 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:26.910549 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5886cab5-48e8-4307-aa42-b8daf64c19f1 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:26.919218 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe37dc1-8f81-4fac-9126-c777b1b7955e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:26.920819 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe37dc1-8f81-4fac-9126-c777b1b7955e None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:26.946214 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb17e366-75a8-4bc4-8994-9bd5914a0561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:26.946834 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17e366-75a8-4bc4-8994-9bd5914a0561 None admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:26.947116 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17e366-75a8-4bc4-8994-9bd5914a0561 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:26.947393 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17e366-75a8-4bc4-8994-9bd5914a0561 None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:26.948097 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb17e366-75a8-4bc4-8994-9bd5914a0561 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:26.949457 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb17e366-75a8-4bc4-8994-9bd5914a0561 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:26.966388 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:27.019848 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 427/864] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:13:26 2022] GET /identity/v3/auth/tokens => generated 1271 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 15:13:27.020713 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe37dc1-8f81-4fac-9126-c777b1b7955e None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:27.055118 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 438/865] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 5 15:13:26 2022] POST /identity/v3/services => generated 251 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:13:27.056269 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb17e366-75a8-4bc4-8994-9bd5914a0561 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:27.142401 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe37dc1-8f81-4fac-9126-c777b1b7955e None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:27.199498 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42fccfe-690c-4d6e-a78e-5616049a773b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:27.203795 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42fccfe-690c-4d6e-a78e-5616049a773b None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:27.204407 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42fccfe-690c-4d6e-a78e-5616049a773b None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:27.205264 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42fccfe-690c-4d6e-a78e-5616049a773b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:27.217018 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb17e366-75a8-4bc4-8994-9bd5914a0561 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:27.222497 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:27.254358 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36443ad-c2b6-4c8a-b80c-619d135d8e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:27.255003 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36443ad-c2b6-4c8a-b80c-619d135d8e43 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:27.255277 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36443ad-c2b6-4c8a-b80c-619d135d8e43 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:27.255552 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36443ad-c2b6-4c8a-b80c-619d135d8e43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:27.268460 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36443ad-c2b6-4c8a-b80c-619d135d8e43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:27.269919 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36443ad-c2b6-4c8a-b80c-619d135d8e43 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:27.271693 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:27.288207 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-d36443ad-c2b6-4c8a-b80c-619d135d8e43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:27.288207 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:27.288207 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:27.288207 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:27.288207 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:27.288207 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:27.288207 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:27.288207 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:27.288207 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:27.288207 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:27.288207 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:27.288207 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:27.288207 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:27.288207 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:27.288207 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:27.289163 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:27.289163 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:27.289163 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:27.289163 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:27.289163 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:27.289163 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:27.289163 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:27.289163 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:27.289163 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:27.289163 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:27.289163 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:27.289163 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:27.289163 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:27.289163 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:27.289163 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:27.289163 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:27.289163 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:27.290721 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:27.290721 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:27.290721 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:27.290721 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:27.290721 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 439/866] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:27.294824 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36443ad-c2b6-4c8a-b80c-619d135d8e43 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:27.296824 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42fccfe-690c-4d6e-a78e-5616049a773b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:27.300391 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42fccfe-690c-4d6e-a78e-5616049a773b None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:27.426272 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36443ad-c2b6-4c8a-b80c-619d135d8e43 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:27.433274 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 428/867] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:13:27 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 15:13:27.435212 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42fccfe-690c-4d6e-a78e-5616049a773b None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:27.471009 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d62d86c-30c8-46cf-bd72-7443b5af9292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:27.472278 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62d86c-30c8-46cf-bd72-7443b5af9292 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:27.473057 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62d86c-30c8-46cf-bd72-7443b5af9292 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:27.473801 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62d86c-30c8-46cf-bd72-7443b5af9292 None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:27.474554 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d62d86c-30c8-46cf-bd72-7443b5af9292 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:27.474861 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d62d86c-30c8-46cf-bd72-7443b5af9292 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:27.477308 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d62d86c-30c8-46cf-bd72-7443b5af9292 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:27.484484 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:27.492889 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 440/868] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:27.500368 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d62d86c-30c8-46cf-bd72-7443b5af9292 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:27.553710 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42fccfe-690c-4d6e-a78e-5616049a773b None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:27.562276 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:27.592419 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4d85e-2a12-408b-970e-af86f61b5255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:27.593314 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4d85e-2a12-408b-970e-af86f61b5255 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:27.593744 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4d85e-2a12-408b-970e-af86f61b5255 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:27.594795 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4d85e-2a12-408b-970e-af86f61b5255 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:27.607625 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4d85e-2a12-408b-970e-af86f61b5255 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:27.609409 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4d85e-2a12-408b-970e-af86f61b5255 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:27.610157 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d62d86c-30c8-46cf-bd72-7443b5af9292 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:27.625937 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-3fe4d85e-2a12-408b-970e-af86f61b5255 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:27.625937 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:27.625937 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:27.625937 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:27.625937 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:27.625937 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:27.625937 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:27.625937 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:27.625937 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:27.625937 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:27.625937 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:27.625937 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:27.625937 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:27.625937 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:27.625937 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:27.626881 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:27.626881 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:27.626881 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:27.626881 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:27.626881 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:27.626881 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:27.626881 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:27.626881 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:27.626881 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:27.626881 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:27.626881 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:27.626881 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:27.626881 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:27.626881 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:27.626881 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:27.626881 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:27.626881 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:27.627776 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:27.627776 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:27.627776 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:27.627776 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:27.630031 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 429/869] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:13:27.637478 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4d85e-2a12-408b-970e-af86f61b5255 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:27.652239 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f242e34-d027-4505-a2fa-4e2d51dd4215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:27.653063 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f242e34-d027-4505-a2fa-4e2d51dd4215 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:27.653464 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f242e34-d027-4505-a2fa-4e2d51dd4215 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:27.653875 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f242e34-d027-4505-a2fa-4e2d51dd4215 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:27.661741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f242e34-d027-4505-a2fa-4e2d51dd4215 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:27.663385 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f242e34-d027-4505-a2fa-4e2d51dd4215 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:27.675221 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-7f242e34-d027-4505-a2fa-4e2d51dd4215 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:27.675221 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:27.675221 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:27.675221 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:27.675221 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:27.675221 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:27.675221 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:27.675221 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:27.675221 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:27.675221 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:27.675221 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:27.675221 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:27.675221 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 15:13:27.675221 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 15:13:27.675221 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 15:13:27.676319 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 15:13:27.676319 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:27.676319 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:27.676319 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:27.676319 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:27.676319 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:27.676319 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:27.676319 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:27.676319 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:27.676319 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:27.676319 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:27.676319 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:27.676319 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:27.676319 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:27.676319 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:27.676319 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:27.676319 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:27.677212 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 15:13:27.677212 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 15:13:27.677212 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 15:13:27.677212 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 15:13:27.677212 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 15:13:27.677212 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 15:13:27.677212 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:27.677212 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:27.681885 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 441/870] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:13:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:27.683550 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f242e34-d027-4505-a2fa-4e2d51dd4215 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:27.746772 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4d85e-2a12-408b-970e-af86f61b5255 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:27.780475 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:27.785264 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b88a8-137d-4037-8560-5fb146d350d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:27.786184 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b88a8-137d-4037-8560-5fb146d350d3 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:27.786643 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b88a8-137d-4037-8560-5fb146d350d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:27.787150 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b88a8-137d-4037-8560-5fb146d350d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:27.788008 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b88a8-137d-4037-8560-5fb146d350d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:27.788528 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b88a8-137d-4037-8560-5fb146d350d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:27.790029 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f242e34-d027-4505-a2fa-4e2d51dd4215 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:27.790442 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b88a8-137d-4037-8560-5fb146d350d3 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:27.806640 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 430/871] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:27.807183 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:27.810180 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b88a8-137d-4037-8560-5fb146d350d3 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:27.826893 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5546e1-6c0d-4b08-bb57-d76958495f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:27.827810 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5546e1-6c0d-4b08-bb57-d76958495f56 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:27.829083 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5546e1-6c0d-4b08-bb57-d76958495f56 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:27.829232 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5546e1-6c0d-4b08-bb57-d76958495f56 None admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:27.829988 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5546e1-6c0d-4b08-bb57-d76958495f56 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:27.830370 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5546e1-6c0d-4b08-bb57-d76958495f56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:27.832173 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5546e1-6c0d-4b08-bb57-d76958495f56 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:27.849939 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 442/872] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 15:13:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:27.851352 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5546e1-6c0d-4b08-bb57-d76958495f56 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:27.928668 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b88a8-137d-4037-8560-5fb146d350d3 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:27.961294 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5546e1-6c0d-4b08-bb57-d76958495f56 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:27.981296 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5145f2f-87fb-4a29-8fb3-f7a8081f6299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:27.981946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5145f2f-87fb-4a29-8fb3-f7a8081f6299 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:27.982229 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5145f2f-87fb-4a29-8fb3-f7a8081f6299 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:27.982509 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5145f2f-87fb-4a29-8fb3-f7a8081f6299 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:27.992239 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5145f2f-87fb-4a29-8fb3-f7a8081f6299 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:27.994658 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5145f2f-87fb-4a29-8fb3-f7a8081f6299 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:27.998763 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94813745-037f-4953-ab5b-3d53b468f326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:27.999723 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94813745-037f-4953-ab5b-3d53b468f326 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:28.000774 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94813745-037f-4953-ab5b-3d53b468f326 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:28.001197 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94813745-037f-4953-ab5b-3d53b468f326 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:28.002095 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94813745-037f-4953-ab5b-3d53b468f326 None admin] RBAC: Adding query filter params (role.id=266a023f66e043f99efe2941fe58d481, scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83, user.id=1654fb6a81e14d6b99ab495bb6d029d7) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:28.002513 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94813745-037f-4953-ab5b-3d53b468f326 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:28.004127 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94813745-037f-4953-ab5b-3d53b468f326 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:28.008177 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-a5145f2f-87fb-4a29-8fb3-f7a8081f6299 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:28.008177 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:28.008177 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:28.008177 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:28.008177 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:28.008177 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:28.008177 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:28.008177 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:28.008177 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:28.008177 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:28.008177 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:28.008177 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:28.008177 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 15:13:28.008177 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 15:13:28.008177 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 15:13:28.009228 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 15:13:28.009228 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:28.009228 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:28.009228 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:28.009228 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:28.009228 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:28.009228 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:28.009228 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:28.009228 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:28.009228 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:28.009228 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:28.009228 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:28.009228 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:28.009228 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:28.009228 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:28.009228 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:28.009228 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:28.010436 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 15:13:28.010436 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 15:13:28.010436 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 15:13:28.010436 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 15:13:28.010436 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 15:13:28.010436 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 15:13:28.010436 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:28.010436 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:28.010436 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 431/873] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:13:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:28.016508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 443/874] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:13:27 2022] GET /identity/v3/role_assignments?user.id=1654fb6a81e14d6b99ab495bb6d029d7&scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83&role.id=266a023f66e043f99efe2941fe58d481 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:28.017517 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5145f2f-87fb-4a29-8fb3-f7a8081f6299 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:28.036726 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:28.140109 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5145f2f-87fb-4a29-8fb3-f7a8081f6299 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:28.176289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1c4e0-42ef-4861-b398-1585f186a432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:28.176958 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1c4e0-42ef-4861-b398-1585f186a432 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:28.177208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1c4e0-42ef-4861-b398-1585f186a432 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:28.177487 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1c4e0-42ef-4861-b398-1585f186a432 None admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:28.178325 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef1c4e0-42ef-4861-b398-1585f186a432 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:28.178601 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef1c4e0-42ef-4861-b398-1585f186a432 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:28.180055 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef1c4e0-42ef-4861-b398-1585f186a432 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:28.193171 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 432/875] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 15:13:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:28.198827 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94813745-037f-4953-ab5b-3d53b468f326 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:28.243226 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:28.309488 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94813745-037f-4953-ab5b-3d53b468f326 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:28.344741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f9cda-8217-43a8-a211-cc75f413895c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:28.345365 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f9cda-8217-43a8-a211-cc75f413895c None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:28.345650 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f9cda-8217-43a8-a211-cc75f413895c None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:28.345928 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f9cda-8217-43a8-a211-cc75f413895c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:28.346727 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f9cda-8217-43a8-a211-cc75f413895c None admin] RBAC: Adding query filter params (role.id=7c88c815db854954b516ab97cd471647, scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83, user.id=85154ce7446245738edcc8702dbf6861) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:28.347042 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f9cda-8217-43a8-a211-cc75f413895c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:28.348514 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f9cda-8217-43a8-a211-cc75f413895c None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:28.354912 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 444/876] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:13:28 2022] GET /identity/v3/role_assignments?user.id=85154ce7446245738edcc8702dbf6861&scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83&role.id=7c88c815db854954b516ab97cd471647 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:28.496419 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a23ba8-e18b-4531-82b0-9ba90e1cd166 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:28.496842 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a23ba8-e18b-4531-82b0-9ba90e1cd166 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:28.497497 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a23ba8-e18b-4531-82b0-9ba90e1cd166 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:28.498973 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 433/877] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:28.504482 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454ffd0-77ae-4c12-b166-1da6902e167b None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:28.504773 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454ffd0-77ae-4c12-b166-1da6902e167b None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:28.505069 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454ffd0-77ae-4c12-b166-1da6902e167b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:28.524943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-6454ffd0-77ae-4c12-b166-1da6902e167b None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:28.526202 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-6454ffd0-77ae-4c12-b166-1da6902e167b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:28.603473 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 445/878] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:28 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:13:28.627192 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a23ba8-e18b-4531-82b0-9ba90e1cd166 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:28.705469 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:28.738037 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a23ba8-e18b-4531-82b0-9ba90e1cd166 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:28.776750 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115410b2-85d4-4d84-ba0d-62b5c9d2fa98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:28.777656 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115410b2-85d4-4d84-ba0d-62b5c9d2fa98 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:28.778118 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115410b2-85d4-4d84-ba0d-62b5c9d2fa98 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:28.778562 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115410b2-85d4-4d84-ba0d-62b5c9d2fa98 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:28.779341 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115410b2-85d4-4d84-ba0d-62b5c9d2fa98 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:28.780872 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115410b2-85d4-4d84-ba0d-62b5c9d2fa98 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:28.790905 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-115410b2-85d4-4d84-ba0d-62b5c9d2fa98 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 15:13:28.790905 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:28.790905 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 15:13:28.790905 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 15:13:28.790905 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 15:13:28.790905 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 15:13:28.790905 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 15:13:28.790905 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:28.790905 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 15:13:28.790905 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:28.790905 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 15:13:28.790905 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:28.790905 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:28.790905 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:28.790905 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:28.790905 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:28.792051 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:28.792051 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:28.792051 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:28.792051 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:28.792051 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:28.792051 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:28.792051 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:28.792051 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 15:13:28.792051 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 15:13:28.792051 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 15:13:28.792051 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 15:13:28.792051 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:28.792051 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:28.792051 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:28.792051 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:28.792051 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:28.793141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:28.793141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:28.793141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:28.793141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:28.793141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:28.793141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:28.793141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:28.793141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:28.793141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:28.793141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:28.793141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:28.793141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 15:13:28.793141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:28.793141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 15:13:28.793141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:28.795687 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 434/879] 10.222.0.40 () {56 vars in 1132 bytes} [Fri Aug 5 15:13:28 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:28.802082 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454ffd0-77ae-4c12-b166-1da6902e167b None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:28.833982 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:28.908437 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454ffd0-77ae-4c12-b166-1da6902e167b None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:28.943756 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c9707e-19b5-4d3a-b91d-a5948aac8efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:28.944414 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9707e-19b5-4d3a-b91d-a5948aac8efb None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:28.944692 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9707e-19b5-4d3a-b91d-a5948aac8efb None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:28.944968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9707e-19b5-4d3a-b91d-a5948aac8efb None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:28.945645 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c9707e-19b5-4d3a-b91d-a5948aac8efb None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:28.945948 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c9707e-19b5-4d3a-b91d-a5948aac8efb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:28.947308 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c9707e-19b5-4d3a-b91d-a5948aac8efb None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:28.952439 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 446/880] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 5 15:13:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:28.958684 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115410b2-85d4-4d84-ba0d-62b5c9d2fa98 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:29.075114 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115410b2-85d4-4d84-ba0d-62b5c9d2fa98 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:29.111273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30486857-fd76-4d38-9e4b-ac30afa68649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:29.111895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30486857-fd76-4d38-9e4b-ac30afa68649 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:29.112206 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30486857-fd76-4d38-9e4b-ac30afa68649 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:29.112482 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30486857-fd76-4d38-9e4b-ac30afa68649 None admin] PATH_INFO: `/v3/services` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:29.113154 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30486857-fd76-4d38-9e4b-ac30afa68649 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:29.113459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30486857-fd76-4d38-9e4b-ac30afa68649 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:29.114817 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30486857-fd76-4d38-9e4b-ac30afa68649 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:29.116923 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:29.121054 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 435/881] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 5 15:13:28 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:29.127333 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c9707e-19b5-4d3a-b91d-a5948aac8efb None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:29.151296 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96bd19-8339-4939-8aa2-fb646570ed44 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:29.151589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96bd19-8339-4939-8aa2-fb646570ed44 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:29.151873 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96bd19-8339-4939-8aa2-fb646570ed44 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:29.153502 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 436/882] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:29.158973 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d6bc5-1a76-4222-8d10-7e6cb7cc5753 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:29.159263 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d6bc5-1a76-4222-8d10-7e6cb7cc5753 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:29.159530 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d6bc5-1a76-4222-8d10-7e6cb7cc5753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:29.180641 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-972d6bc5-1a76-4222-8d10-7e6cb7cc5753 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:29.182034 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-972d6bc5-1a76-4222-8d10-7e6cb7cc5753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:29.231368 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:29.238402 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c9707e-19b5-4d3a-b91d-a5948aac8efb None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:29.272208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 437/883] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:29 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 15:13:29.283386 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6240511-f94a-46fe-bfcc-c68ea902c76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:29.284420 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6240511-f94a-46fe-bfcc-c68ea902c76d None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:29.284932 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6240511-f94a-46fe-bfcc-c68ea902c76d None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:29.285456 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6240511-f94a-46fe-bfcc-c68ea902c76d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:29.286414 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6240511-f94a-46fe-bfcc-c68ea902c76d None admin] RBAC: Adding query filter params (interface=public, service_id=f60b1c21610b433287cb0d93cc21eaca, region_id=RegionOne) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:29.286931 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6240511-f94a-46fe-bfcc-c68ea902c76d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:29.288623 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6240511-f94a-46fe-bfcc-c68ea902c76d None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:29.292001 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972d6bc5-1a76-4222-8d10-7e6cb7cc5753 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:29.301334 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 447/884] 10.222.0.40 () {56 vars in 1251 bytes} [Fri Aug 5 15:13:29 2022] GET /identity/v3/endpoints?service_id=f60b1c21610b433287cb0d93cc21eaca&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:29.305871 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6240511-f94a-46fe-bfcc-c68ea902c76d None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:29.410737 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972d6bc5-1a76-4222-8d10-7e6cb7cc5753 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:29.425950 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6240511-f94a-46fe-bfcc-c68ea902c76d None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:29.443236 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:29.449601 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9369aa58-eed1-436e-8e22-6ff01d4f67a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:29.450300 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369aa58-eed1-436e-8e22-6ff01d4f67a1 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:29.450580 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369aa58-eed1-436e-8e22-6ff01d4f67a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:29.450856 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369aa58-eed1-436e-8e22-6ff01d4f67a1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:29.451534 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9369aa58-eed1-436e-8e22-6ff01d4f67a1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:29.452960 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9369aa58-eed1-436e-8e22-6ff01d4f67a1 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:29.462634 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-9369aa58-eed1-436e-8e22-6ff01d4f67a1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 15:13:29.462634 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:29.462634 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 15:13:29.462634 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 15:13:29.462634 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 15:13:29.462634 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 15:13:29.462634 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 15:13:29.462634 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:29.462634 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 15:13:29.462634 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:29.462634 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 15:13:29.462634 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:29.462634 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:29.462634 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:29.462634 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:29.462634 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:29.462634 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:29.463643 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:29.463643 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:29.463643 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:29.463643 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:29.463643 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:29.463643 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:29.463643 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 15:13:29.463643 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 15:13:29.463643 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 15:13:29.463643 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 15:13:29.463643 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:29.463643 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:29.463643 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:29.463643 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:29.463643 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:29.463643 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:29.464514 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:29.464514 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:29.464514 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:29.464514 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:29.464514 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:29.464514 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:29.464514 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:29.464514 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:29.464514 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:29.464514 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:29.464514 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 15:13:29.464514 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:29.464514 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 15:13:29.464514 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:29.465258 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c7f7db-f898-48ac-82b3-ee55ca26542e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:29.465349 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 438/885] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 15:13:29 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:13:29.465419 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c7f7db-f898-48ac-82b3-ee55ca26542e None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:29.465419 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c7f7db-f898-48ac-82b3-ee55ca26542e None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:29.465419 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c7f7db-f898-48ac-82b3-ee55ca26542e None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:29.468937 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c7f7db-f898-48ac-82b3-ee55ca26542e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:29.468937 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c7f7db-f898-48ac-82b3-ee55ca26542e None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:29.470218 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9369aa58-eed1-436e-8e22-6ff01d4f67a1 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:29.473864 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 448/886] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Aug 5 15:13:29 2022] GET /identity/v3/services => generated 1268 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:13:29.590946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9369aa58-eed1-436e-8e22-6ff01d4f67a1 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:29.634123 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5883ce21-1b54-4ed8-9e8e-9fe08e407015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:29.634905 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883ce21-1b54-4ed8-9e8e-9fe08e407015 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:29.635066 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883ce21-1b54-4ed8-9e8e-9fe08e407015 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:29.635320 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883ce21-1b54-4ed8-9e8e-9fe08e407015 None admin] PATH_INFO: `/v3/services` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:29.636057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5883ce21-1b54-4ed8-9e8e-9fe08e407015 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:29.636393 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5883ce21-1b54-4ed8-9e8e-9fe08e407015 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:29.637817 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5883ce21-1b54-4ed8-9e8e-9fe08e407015 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:29.643330 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 439/887] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 15:13:29 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:29.648935 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c7f7db-f898-48ac-82b3-ee55ca26542e None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:29.677034 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:29.757185 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c7f7db-f898-48ac-82b3-ee55ca26542e None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:29.792825 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2451afd-b781-4d61-9a50-9bfe244756fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:29.793451 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2451afd-b781-4d61-9a50-9bfe244756fb None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:29.793735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2451afd-b781-4d61-9a50-9bfe244756fb None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:29.794012 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2451afd-b781-4d61-9a50-9bfe244756fb None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:29.794690 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2451afd-b781-4d61-9a50-9bfe244756fb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:29.794998 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2451afd-b781-4d61-9a50-9bfe244756fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:29.796400 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2451afd-b781-4d61-9a50-9bfe244756fb None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:29.802326 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 449/888] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 15:13:29 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:29.809201 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5883ce21-1b54-4ed8-9e8e-9fe08e407015 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:29.871127 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:29.916791 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5883ce21-1b54-4ed8-9e8e-9fe08e407015 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:29.952496 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab45a38-a26c-4687-8488-c111f82b3c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:29.953035 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab45a38-a26c-4687-8488-c111f82b3c4d None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:29.953358 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab45a38-a26c-4687-8488-c111f82b3c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:29.953635 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab45a38-a26c-4687-8488-c111f82b3c4d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:29.954389 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab45a38-a26c-4687-8488-c111f82b3c4d None admin] RBAC: Adding query filter params (interface=public, service_id=f377a790c0e74e1e8ea4d0004ac9d279, region_id=RegionOne) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:29.954698 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab45a38-a26c-4687-8488-c111f82b3c4d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:29.956050 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab45a38-a26c-4687-8488-c111f82b3c4d None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:29.961836 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 440/889] 10.222.0.40 () {56 vars in 1251 bytes} [Fri Aug 5 15:13:29 2022] GET /identity/v3/endpoints?service_id=f377a790c0e74e1e8ea4d0004ac9d279&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:29.969732 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2451afd-b781-4d61-9a50-9bfe244756fb None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:30.066485 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:30.079973 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2451afd-b781-4d61-9a50-9bfe244756fb None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:30.117007 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10c94c9-465b-42f7-92de-8610a86dc0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:30.117956 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c94c9-465b-42f7-92de-8610a86dc0f7 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:30.118423 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c94c9-465b-42f7-92de-8610a86dc0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:30.118897 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c94c9-465b-42f7-92de-8610a86dc0f7 None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:30.119721 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10c94c9-465b-42f7-92de-8610a86dc0f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:30.121387 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10c94c9-465b-42f7-92de-8610a86dc0f7 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:30.128846 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 450/890] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Aug 5 15:13:29 2022] GET /identity/v3/services => generated 1268 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:13:30.272220 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2db52-8368-413d-b6b8-9382a1df5b9d None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:30.272953 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2db52-8368-413d-b6b8-9382a1df5b9d None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:30.273440 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2db52-8368-413d-b6b8-9382a1df5b9d None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:30.275251 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 441/891] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:30.282385 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ecedf-09e5-4511-8d1f-92d7df1e3319 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:30.282994 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ecedf-09e5-4511-8d1f-92d7df1e3319 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:30.283491 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ecedf-09e5-4511-8d1f-92d7df1e3319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:30.305052 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-2f1ecedf-09e5-4511-8d1f-92d7df1e3319 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:30.306647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-2f1ecedf-09e5-4511-8d1f-92d7df1e3319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:30.377784 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 451/892] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:30 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 15:13:30.401438 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a2db52-8368-413d-b6b8-9382a1df5b9d None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:30.511051 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a2db52-8368-413d-b6b8-9382a1df5b9d None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:30.536466 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:30.549215 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc83d7-1b4f-4bce-96a3-ce5a9754c520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:30.550109 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc83d7-1b4f-4bce-96a3-ce5a9754c520 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:30.550733 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc83d7-1b4f-4bce-96a3-ce5a9754c520 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:30.561625 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc83d7-1b4f-4bce-96a3-ce5a9754c520 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:30.562587 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc83d7-1b4f-4bce-96a3-ce5a9754c520 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:13:30.567966 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-9dbc83d7-1b4f-4bce-96a3-ce5a9754c520 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 15:13:30.567966 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:30.567966 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:30.567966 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:30.567966 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:30.567966 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:30.567966 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:30.567966 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:30.567966 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:30.567966 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:30.567966 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:30.567966 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:30.567966 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:13:30.567966 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:13:30.567966 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:13:30.567966 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:13:30.568911 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:13:30.568911 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:13:30.568911 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:13:30.568911 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:13:30.568911 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:30.568911 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:30.568911 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:30.568911 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:30.568911 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:30.568911 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:30.568911 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:30.568911 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:30.568911 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:30.568911 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:30.568911 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:30.568911 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:30.569765 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:30.569765 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:30.569765 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:30.569765 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:30.569765 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:13:30.569765 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:13:30.569765 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:13:30.569765 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:13:30.569765 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:13:30.569765 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:13:30.569765 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 15:13:30.569765 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:30.572682 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 442/893] 10.222.0.40 () {56 vars in 1120 bytes} [Fri Aug 5 15:13:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:30.576561 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ddb34-62d8-4db1-a115-5015d051801a None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:30.576561 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ddb34-62d8-4db1-a115-5015d051801a None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:30.576561 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ddb34-62d8-4db1-a115-5015d051801a None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:30.580056 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 452/894] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 15:13:30.581688 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc83d7-1b4f-4bce-96a3-ce5a9754c520 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:30.582141 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34783ee-76dd-4ff0-a45a-45eeeeb02019 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:30.582441 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34783ee-76dd-4ff0-a45a-45eeeeb02019 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:30.582719 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34783ee-76dd-4ff0-a45a-45eeeeb02019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:30.605604 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-d34783ee-76dd-4ff0-a45a-45eeeeb02019 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:30.608152 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-d34783ee-76dd-4ff0-a45a-45eeeeb02019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:30.698238 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 453/895] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:30 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:13:30.701214 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc83d7-1b4f-4bce-96a3-ce5a9754c520 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:30.720506 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34783ee-76dd-4ff0-a45a-45eeeeb02019 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:30.735675 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:30.740645 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5fc1b8-6d76-4fa1-af60-4ff6c663e000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:30.741499 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fc1b8-6d76-4fa1-af60-4ff6c663e000 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:30.742033 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fc1b8-6d76-4fa1-af60-4ff6c663e000 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:30.742505 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fc1b8-6d76-4fa1-af60-4ff6c663e000 None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:30.743592 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5fc1b8-6d76-4fa1-af60-4ff6c663e000 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:30.744118 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5fc1b8-6d76-4fa1-af60-4ff6c663e000 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:30.745630 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5fc1b8-6d76-4fa1-af60-4ff6c663e000 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:30.755439 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 443/896] 10.222.0.40 () {56 vars in 1115 bytes} [Fri Aug 5 15:13:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:30.761061 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5fc1b8-6d76-4fa1-af60-4ff6c663e000 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:30.807668 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:30.834372 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34783ee-76dd-4ff0-a45a-45eeeeb02019 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:30.866921 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5fc1b8-6d76-4fa1-af60-4ff6c663e000 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:30.871452 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ad903-622d-4217-a261-82594f0a13aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:30.872145 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ad903-622d-4217-a261-82594f0a13aa None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:30.872700 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ad903-622d-4217-a261-82594f0a13aa None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:30.872976 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ad903-622d-4217-a261-82594f0a13aa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:30.873607 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912ad903-622d-4217-a261-82594f0a13aa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:30.875504 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912ad903-622d-4217-a261-82594f0a13aa None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:30.887908 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-912ad903-622d-4217-a261-82594f0a13aa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 15:13:30.887908 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:30.887908 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 15:13:30.887908 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 15:13:30.887908 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 15:13:30.887908 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 15:13:30.887908 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 15:13:30.887908 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:30.887908 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 15:13:30.887908 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:30.887908 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 15:13:30.887908 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:30.887908 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:30.887908 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:30.887908 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:30.887908 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:30.887908 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:30.888979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:30.888979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:30.888979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:30.888979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:30.888979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:30.888979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:30.888979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 15:13:30.888979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 15:13:30.888979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 15:13:30.888979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 15:13:30.888979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:30.888979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:30.888979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:30.888979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:30.888979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:30.888979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:30.889840 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:30.889840 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:30.889840 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:30.889840 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:30.889840 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:30.889840 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:30.889840 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:30.889840 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:30.889840 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:30.889840 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:30.889840 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 15:13:30.889840 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:30.889840 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 15:13:30.889840 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:30.895073 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 454/897] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 5 15:13:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:13:30.897001 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ad903-622d-4217-a261-82594f0a13aa None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:30.910665 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b051ddd-4eaa-411e-978b-d593f679d85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:30.911772 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b051ddd-4eaa-411e-978b-d593f679d85b None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:30.912307 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b051ddd-4eaa-411e-978b-d593f679d85b None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:30.912921 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b051ddd-4eaa-411e-978b-d593f679d85b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:30.925725 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b051ddd-4eaa-411e-978b-d593f679d85b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:30.927394 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b051ddd-4eaa-411e-978b-d593f679d85b None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:30.943505 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-8b051ddd-4eaa-411e-978b-d593f679d85b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:30.943505 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:30.943505 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:30.943505 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:30.943505 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:30.943505 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:30.943505 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:30.943505 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:30.943505 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:30.943505 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:30.943505 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:30.943505 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:30.943505 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:30.943505 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:30.943505 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:30.944679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:30.944679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:30.944679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:30.944679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:30.944679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:30.944679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:30.944679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:30.944679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:30.944679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:30.944679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:30.944679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:30.944679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:30.944679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:30.944679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:30.944679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:30.944679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:30.944679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:30.945779 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:30.945779 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:30.945779 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:30.945779 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:30.951496 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 444/898] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:30.952968 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b051ddd-4eaa-411e-978b-d593f679d85b None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:31.018270 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ad903-622d-4217-a261-82594f0a13aa None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:31.032421 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:31.053437 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f61faa0-c3b9-4f08-96a3-9d60e759d135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:31.054063 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61faa0-c3b9-4f08-96a3-9d60e759d135 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:31.054358 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61faa0-c3b9-4f08-96a3-9d60e759d135 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:31.054643 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61faa0-c3b9-4f08-96a3-9d60e759d135 None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:31.055319 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f61faa0-c3b9-4f08-96a3-9d60e759d135 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:31.055633 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f61faa0-c3b9-4f08-96a3-9d60e759d135 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:31.056236 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:31.057086 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f61faa0-c3b9-4f08-96a3-9d60e759d135 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:31.067087 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 455/899] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Aug 5 15:13:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:31.068349 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b051ddd-4eaa-411e-978b-d593f679d85b None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:31.069173 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f61faa0-c3b9-4f08-96a3-9d60e759d135 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:31.112751 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4e75d-13af-4a07-84a4-77669b81a31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:31.113373 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4e75d-13af-4a07-84a4-77669b81a31e None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:31.113660 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4e75d-13af-4a07-84a4-77669b81a31e None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:31.113937 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4e75d-13af-4a07-84a4-77669b81a31e None admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:31.114626 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a4e75d-13af-4a07-84a4-77669b81a31e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:31.114937 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a4e75d-13af-4a07-84a4-77669b81a31e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:31.116338 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a4e75d-13af-4a07-84a4-77669b81a31e None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:31.134185 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 445/900] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:31.135544 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4e75d-13af-4a07-84a4-77669b81a31e None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:31.192004 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f61faa0-c3b9-4f08-96a3-9d60e759d135 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:31.228516 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfefbfcb-b0eb-4452-a04f-5adb6d5a016f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:31.229536 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfefbfcb-b0eb-4452-a04f-5adb6d5a016f None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:31.230026 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfefbfcb-b0eb-4452-a04f-5adb6d5a016f None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:31.230508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfefbfcb-b0eb-4452-a04f-5adb6d5a016f None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:31.231408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfefbfcb-b0eb-4452-a04f-5adb6d5a016f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:31.231920 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfefbfcb-b0eb-4452-a04f-5adb6d5a016f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:31.233580 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfefbfcb-b0eb-4452-a04f-5adb6d5a016f None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:31.239456 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 456/901] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Aug 5 15:13:31 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:31.244582 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4e75d-13af-4a07-84a4-77669b81a31e None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:31.283937 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b7834-869c-481a-8869-9cbee1bcf4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:31.285021 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b7834-869c-481a-8869-9cbee1bcf4bc None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:31.285524 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b7834-869c-481a-8869-9cbee1bcf4bc None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:31.286005 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b7834-869c-481a-8869-9cbee1bcf4bc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:31.294675 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8b7834-869c-481a-8869-9cbee1bcf4bc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:31.296569 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8b7834-869c-481a-8869-9cbee1bcf4bc None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:31.311289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-ce8b7834-869c-481a-8869-9cbee1bcf4bc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 15:13:31.311289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:31.311289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:31.311289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:31.311289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:31.311289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:31.311289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:31.311289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:31.311289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:31.311289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:31.311289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:31.311289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:31.311289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 15:13:31.311289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 15:13:31.311289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 15:13:31.311289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 15:13:31.312341 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:31.312341 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:31.312341 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 15:13:31.312341 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:31.312341 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 15:13:31.312341 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:31.312341 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:31.312341 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:31.312341 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:31.312341 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:31.312341 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:31.312341 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:31.312341 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:31.312341 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:31.312341 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:31.312341 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:31.312341 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:31.312341 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:31.313938 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:31.313938 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:31.313938 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 15:13:31.313938 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 15:13:31.313938 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 15:13:31.313938 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 15:13:31.313938 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 15:13:31.313938 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 15:13:31.313938 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 15:13:31.313938 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:31.316834 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 446/902] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Aug 5 15:13:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:31.322384 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfefbfcb-b0eb-4452-a04f-5adb6d5a016f None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:31.326886 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:31.439810 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfefbfcb-b0eb-4452-a04f-5adb6d5a016f None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:31.483068 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6535bbc-bd3c-4179-8fe8-b155e17e9869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:31.483841 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6535bbc-bd3c-4179-8fe8-b155e17e9869 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:31.484324 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6535bbc-bd3c-4179-8fe8-b155e17e9869 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:31.485214 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6535bbc-bd3c-4179-8fe8-b155e17e9869 None admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:31.486123 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6535bbc-bd3c-4179-8fe8-b155e17e9869 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:31.486558 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6535bbc-bd3c-4179-8fe8-b155e17e9869 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:31.488115 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6535bbc-bd3c-4179-8fe8-b155e17e9869 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:31.506912 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 457/903] 10.222.0.40 () {56 vars in 1149 bytes} [Fri Aug 5 15:13:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:31.512737 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b7834-869c-481a-8869-9cbee1bcf4bc None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:31.533586 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:31.533836 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee614604-7bdd-483b-98bb-2582e3611171 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:31.534136 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee614604-7bdd-483b-98bb-2582e3611171 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:31.534418 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee614604-7bdd-483b-98bb-2582e3611171 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:31.536018 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 458/904] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 15:13:31.540952 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ebb5e-f3bd-46cd-b4f0-2e690885c4c6 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:31.541240 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ebb5e-f3bd-46cd-b4f0-2e690885c4c6 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:31.541526 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ebb5e-f3bd-46cd-b4f0-2e690885c4c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:31.564037 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-3f9ebb5e-f3bd-46cd-b4f0-2e690885c4c6 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:31.565755 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-3f9ebb5e-f3bd-46cd-b4f0-2e690885c4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:31.630226 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b7834-869c-481a-8869-9cbee1bcf4bc None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:31.645844 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:31.668024 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 459/905] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:31 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 15:13:31.679539 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad962edd-1985-41e1-9aec-aeae31d94b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:31.681931 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad962edd-1985-41e1-9aec-aeae31d94b3c None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:31.682468 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad962edd-1985-41e1-9aec-aeae31d94b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:31.682967 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad962edd-1985-41e1-9aec-aeae31d94b3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:31.691460 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9ebb5e-f3bd-46cd-b4f0-2e690885c4c6 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:31.773020 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad962edd-1985-41e1-9aec-aeae31d94b3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:31.775290 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad962edd-1985-41e1-9aec-aeae31d94b3c None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:31.802283 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:31.839117 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9ebb5e-f3bd-46cd-b4f0-2e690885c4c6 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:31.886666 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c346b11-d873-4c2e-a603-201673df925d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:31.887250 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c346b11-d873-4c2e-a603-201673df925d None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:31.887540 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c346b11-d873-4c2e-a603-201673df925d None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:31.887823 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c346b11-d873-4c2e-a603-201673df925d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:31.888524 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c346b11-d873-4c2e-a603-201673df925d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:31.890064 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c346b11-d873-4c2e-a603-201673df925d None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:31.900504 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-4c346b11-d873-4c2e-a603-201673df925d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 15:13:31.900504 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:31.900504 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 15:13:31.900504 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 15:13:31.900504 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 15:13:31.900504 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 15:13:31.900504 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 15:13:31.900504 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:31.900504 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 15:13:31.900504 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:31.900504 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 15:13:31.900504 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:31.900504 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:31.900504 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:31.900504 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:31.900504 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:31.901432 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:31.901432 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:31.901432 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:31.901432 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:31.901432 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:31.901432 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:31.901432 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:31.901432 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 15:13:31.901432 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 15:13:31.901432 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 15:13:31.901432 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 15:13:31.901432 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:31.901432 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:31.901432 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:31.901432 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:31.901432 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:31.902266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:31.902266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:31.902266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:31.902266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:31.902266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:31.902266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:31.902266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:31.902266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:31.902266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:31.902266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:31.902266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:31.902266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 15:13:31.902266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:31.902266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 15:13:31.902266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:31.907450 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 460/906] 10.222.0.40 () {56 vars in 1132 bytes} [Fri Aug 5 15:13:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:13:31.908464 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c346b11-d873-4c2e-a603-201673df925d None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:31.914201 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 447/907] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:13:31 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 05 15:13:31.920112 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad962edd-1985-41e1-9aec-aeae31d94b3c None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:31.978279 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:32.031748 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c346b11-d873-4c2e-a603-201673df925d None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:32.059322 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad962edd-1985-41e1-9aec-aeae31d94b3c None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:32.069507 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f654e7f-000d-4aa3-9860-42652628c7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:32.069944 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f654e7f-000d-4aa3-9860-42652628c7b4 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:32.070305 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f654e7f-000d-4aa3-9860-42652628c7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:32.070660 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f654e7f-000d-4aa3-9860-42652628c7b4 None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:32.071415 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f654e7f-000d-4aa3-9860-42652628c7b4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:32.071798 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f654e7f-000d-4aa3-9860-42652628c7b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:32.073452 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f654e7f-000d-4aa3-9860-42652628c7b4 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:32.084160 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 461/908] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 5 15:13:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:32.085132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f654e7f-000d-4aa3-9860-42652628c7b4 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:32.086470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:32.104053 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b1cf4a-0cf0-4bf4-a915-35682f512473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:32.104739 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b1cf4a-0cf0-4bf4-a915-35682f512473 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:32.105025 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b1cf4a-0cf0-4bf4-a915-35682f512473 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:32.105304 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b1cf4a-0cf0-4bf4-a915-35682f512473 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:32.118244 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b1cf4a-0cf0-4bf4-a915-35682f512473 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:32.120198 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b1cf4a-0cf0-4bf4-a915-35682f512473 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:32.140381 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-37b1cf4a-0cf0-4bf4-a915-35682f512473 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:32.140381 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:32.140381 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:32.140381 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:32.140381 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:32.140381 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:32.140381 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:32.140381 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:32.140381 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:32.140381 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:32.140381 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:32.140381 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:32.140381 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:32.140381 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:32.140381 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:32.141972 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:32.141972 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:32.141972 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:32.141972 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:32.141972 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:32.141972 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:32.141972 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:32.141972 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:32.141972 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:32.141972 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:32.141972 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:32.141972 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:32.141972 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:32.141972 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:32.141972 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:32.141972 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:32.141972 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:32.142860 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:32.142860 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:32.142860 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:32.142860 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:32.148152 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 448/909] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:13:32.151482 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b1cf4a-0cf0-4bf4-a915-35682f512473 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:32.223922 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f654e7f-000d-4aa3-9860-42652628c7b4 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:32.247594 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:32.263828 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331e12a-c9db-46d1-a8dc-96b56a8174d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:32.266172 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331e12a-c9db-46d1-a8dc-96b56a8174d4 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:32.266172 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331e12a-c9db-46d1-a8dc-96b56a8174d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:32.266172 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331e12a-c9db-46d1-a8dc-96b56a8174d4 None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:32.266895 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331e12a-c9db-46d1-a8dc-96b56a8174d4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:32.267251 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331e12a-c9db-46d1-a8dc-96b56a8174d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:32.268678 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331e12a-c9db-46d1-a8dc-96b56a8174d4 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:32.273655 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 462/910] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 5 15:13:32 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:32.273894 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b1cf4a-0cf0-4bf4-a915-35682f512473 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:32.274881 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd923d6e-3fed-4fe9-abba-f1f43481e36c None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:32.275177 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd923d6e-3fed-4fe9-abba-f1f43481e36c None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:32.275474 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd923d6e-3fed-4fe9-abba-f1f43481e36c None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:32.277035 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 463/911] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:32.283391 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26066cfb-6879-4df5-8f46-9fea84b4c62a None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:32.283691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26066cfb-6879-4df5-8f46-9fea84b4c62a None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:32.283998 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26066cfb-6879-4df5-8f46-9fea84b4c62a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:32.303359 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-26066cfb-6879-4df5-8f46-9fea84b4c62a None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:32.304577 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-26066cfb-6879-4df5-8f46-9fea84b4c62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:32.311601 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce731c-6482-4c75-9d8c-80c2af091ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:32.313196 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce731c-6482-4c75-9d8c-80c2af091ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:32.313487 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce731c-6482-4c75-9d8c-80c2af091ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:32.313765 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce731c-6482-4c75-9d8c-80c2af091ae5 None admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:32.314464 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bce731c-6482-4c75-9d8c-80c2af091ae5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:32.315138 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bce731c-6482-4c75-9d8c-80c2af091ae5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:32.316876 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bce731c-6482-4c75-9d8c-80c2af091ae5 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:32.329891 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 449/912] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:32.330673 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce731c-6482-4c75-9d8c-80c2af091ae5 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:32.357824 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:32.388207 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 464/913] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:32 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 15:13:32.388207 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26066cfb-6879-4df5-8f46-9fea84b4c62a None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:32.444464 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce731c-6482-4c75-9d8c-80c2af091ae5 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:32.486999 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191f7b05-449b-4b64-bcb8-7344d08f5bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:32.487851 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f7b05-449b-4b64-bcb8-7344d08f5bd9 None admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:32.487965 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f7b05-449b-4b64-bcb8-7344d08f5bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:32.488215 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f7b05-449b-4b64-bcb8-7344d08f5bd9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:32.488923 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191f7b05-449b-4b64-bcb8-7344d08f5bd9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:32.490289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191f7b05-449b-4b64-bcb8-7344d08f5bd9 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:32.507759 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26066cfb-6879-4df5-8f46-9fea84b4c62a None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:32.522589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:32.542472 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d515eb3-6637-4c6f-98a3-225707016928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:32.543171 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d515eb3-6637-4c6f-98a3-225707016928 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:32.543555 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d515eb3-6637-4c6f-98a3-225707016928 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:32.543844 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d515eb3-6637-4c6f-98a3-225707016928 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:32.551137 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d515eb3-6637-4c6f-98a3-225707016928 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:32.552640 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d515eb3-6637-4c6f-98a3-225707016928 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:32.564482 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-2d515eb3-6637-4c6f-98a3-225707016928 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:32.564482 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:32.564482 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:32.564482 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:32.564482 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:32.564482 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:32.564482 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:32.564482 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:32.564482 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:32.564482 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:32.564482 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:32.564482 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:32.564482 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 15:13:32.564482 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 15:13:32.564482 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 15:13:32.565379 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 15:13:32.565379 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:32.565379 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:32.565379 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:32.565379 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:32.565379 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:32.565379 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:32.565379 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:32.565379 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:32.565379 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:32.565379 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:32.565379 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:32.565379 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:32.565379 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:32.565379 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:32.565379 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:32.565379 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:32.566408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 15:13:32.566408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 15:13:32.566408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 15:13:32.566408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 15:13:32.566408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 15:13:32.566408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 15:13:32.566408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:32.566408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:32.566408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 465/914] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:13:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:32.566408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d515eb3-6637-4c6f-98a3-225707016928 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:32.581022 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 450/915] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:13:32 2022] POST /identity/v3/endpoints => generated 347 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:13:32.581995 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191f7b05-449b-4b64-bcb8-7344d08f5bd9 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:32.695266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d515eb3-6637-4c6f-98a3-225707016928 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:32.722984 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:32.745623 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997baed0-da67-4d79-ae9c-1df7244f28c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:32.746302 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997baed0-da67-4d79-ae9c-1df7244f28c2 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:32.746746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997baed0-da67-4d79-ae9c-1df7244f28c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:32.747030 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997baed0-da67-4d79-ae9c-1df7244f28c2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:32.747685 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191f7b05-449b-4b64-bcb8-7344d08f5bd9 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:32.747936 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997baed0-da67-4d79-ae9c-1df7244f28c2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:32.749593 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997baed0-da67-4d79-ae9c-1df7244f28c2 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:32.761089 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-997baed0-da67-4d79-ae9c-1df7244f28c2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 15:13:32.761089 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:32.761089 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 15:13:32.761089 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 15:13:32.761089 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 15:13:32.761089 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 15:13:32.761089 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 15:13:32.761089 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:32.761089 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 15:13:32.761089 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:32.761089 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 15:13:32.761089 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:32.761089 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:32.761089 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:32.761089 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:32.761089 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:32.761089 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:32.766927 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:32.766927 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:32.766927 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:32.766927 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:32.766927 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:32.766927 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:32.766927 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 15:13:32.766927 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 15:13:32.766927 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 15:13:32.766927 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 15:13:32.766927 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:32.766927 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:32.766927 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:32.766927 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:32.766927 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:32.766927 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:32.767770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:32.767770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:32.767770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:32.767770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:32.767770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:32.767770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:32.767770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:32.767770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:32.767770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:32.767770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:32.767770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 15:13:32.767770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:32.767770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 15:13:32.767770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:32.767770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 466/916] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 15:13:32 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:32.772262 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997baed0-da67-4d79-ae9c-1df7244f28c2 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:32.806887 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d958586-f97a-44e2-8eb3-0e279eac44c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:32.807700 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d958586-f97a-44e2-8eb3-0e279eac44c0 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:32.807975 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d958586-f97a-44e2-8eb3-0e279eac44c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:32.808367 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d958586-f97a-44e2-8eb3-0e279eac44c0 None admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:32.809137 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d958586-f97a-44e2-8eb3-0e279eac44c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:32.809461 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d958586-f97a-44e2-8eb3-0e279eac44c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:32.811000 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d958586-f97a-44e2-8eb3-0e279eac44c0 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:32.813611 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:32.828987 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 451/917] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 15:13:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:32.837426 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d958586-f97a-44e2-8eb3-0e279eac44c0 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:32.893235 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997baed0-da67-4d79-ae9c-1df7244f28c2 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:32.931918 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f130f81-fbe5-45d2-973d-b9c8cd5f4378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:32.932747 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f130f81-fbe5-45d2-973d-b9c8cd5f4378 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:32.933186 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f130f81-fbe5-45d2-973d-b9c8cd5f4378 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:32.933576 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f130f81-fbe5-45d2-973d-b9c8cd5f4378 None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:32.934459 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f130f81-fbe5-45d2-973d-b9c8cd5f4378 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:32.934871 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f130f81-fbe5-45d2-973d-b9c8cd5f4378 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:32.936374 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f130f81-fbe5-45d2-973d-b9c8cd5f4378 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:32.941820 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 467/918] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 15:13:32 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:32.949058 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f130f81-fbe5-45d2-973d-b9c8cd5f4378 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:32.971115 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d958586-f97a-44e2-8eb3-0e279eac44c0 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:33.018696 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ba39e-5fdd-4f8a-b6f7-1008d15d1e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:33.019567 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ba39e-5fdd-4f8a-b6f7-1008d15d1e49 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:33.020027 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ba39e-5fdd-4f8a-b6f7-1008d15d1e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:33.020479 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ba39e-5fdd-4f8a-b6f7-1008d15d1e49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:33.021424 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2ba39e-5fdd-4f8a-b6f7-1008d15d1e49 None admin] RBAC: Adding query filter params (role.id=e7638910be904dc388b948d044a90758, scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83, user.id=1654fb6a81e14d6b99ab495bb6d029d7) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:33.021891 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2ba39e-5fdd-4f8a-b6f7-1008d15d1e49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:33.023527 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2ba39e-5fdd-4f8a-b6f7-1008d15d1e49 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:33.029801 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 452/919] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:13:32 2022] GET /identity/v3/role_assignments?user.id=1654fb6a81e14d6b99ab495bb6d029d7&scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83&role.id=e7638910be904dc388b948d044a90758 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:33.069871 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:33.074510 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f130f81-fbe5-45d2-973d-b9c8cd5f4378 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:33.115488 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b9acc3-2519-4d51-82cd-0c657275a7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:33.116167 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b9acc3-2519-4d51-82cd-0c657275a7d8 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:33.116459 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b9acc3-2519-4d51-82cd-0c657275a7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:33.116742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b9acc3-2519-4d51-82cd-0c657275a7d8 None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:33.117449 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b9acc3-2519-4d51-82cd-0c657275a7d8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:33.117761 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b9acc3-2519-4d51-82cd-0c657275a7d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:33.119163 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b9acc3-2519-4d51-82cd-0c657275a7d8 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:33.125927 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 468/920] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 5 15:13:32 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:33.131641 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ba39e-5fdd-4f8a-b6f7-1008d15d1e49 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:33.145359 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:33.249445 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ba39e-5fdd-4f8a-b6f7-1008d15d1e49 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:33.284938 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7dfc7-6ba0-46fa-9391-c0416fde685f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:33.285835 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7dfc7-6ba0-46fa-9391-c0416fde685f None admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:33.286271 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7dfc7-6ba0-46fa-9391-c0416fde685f None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:33.286692 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7dfc7-6ba0-46fa-9391-c0416fde685f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:33.287562 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd7dfc7-6ba0-46fa-9391-c0416fde685f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:33.289088 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd7dfc7-6ba0-46fa-9391-c0416fde685f None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:33.369115 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 453/921] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:13:33 2022] POST /identity/v3/endpoints => generated 346 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:13:33.373987 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9c127-1263-448d-bbaa-8000771e1556 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:33.374161 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9c127-1263-448d-bbaa-8000771e1556 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:33.375036 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9c127-1263-448d-bbaa-8000771e1556 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:33.375836 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 469/922] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:33.381825 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2574b33-1e34-413f-92d9-ea66ac98bccb None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:33.382145 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2574b33-1e34-413f-92d9-ea66ac98bccb None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:33.382405 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2574b33-1e34-413f-92d9-ea66ac98bccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:33.403138 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-e2574b33-1e34-413f-92d9-ea66ac98bccb None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:33.404417 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-e2574b33-1e34-413f-92d9-ea66ac98bccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:33.474948 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:33.491121 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 454/923] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:33 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:13:33.516468 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b9c127-1263-448d-bbaa-8000771e1556 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:33.629034 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b9c127-1263-448d-bbaa-8000771e1556 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:33.665544 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb5ae49-eb07-411a-9893-ca57a2f29fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:33.666466 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb5ae49-eb07-411a-9893-ca57a2f29fb6 None admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:33.666971 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb5ae49-eb07-411a-9893-ca57a2f29fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:33.667433 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb5ae49-eb07-411a-9893-ca57a2f29fb6 None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:33.668335 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb5ae49-eb07-411a-9893-ca57a2f29fb6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:33.669895 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb5ae49-eb07-411a-9893-ca57a2f29fb6 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:33.759408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 470/924] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 5 15:13:33 2022] POST /identity/v3/services => generated 241 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:13:34.765025 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d8e0c-e178-4f75-ba74-dfe02ff31224 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:34.765604 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d8e0c-e178-4f75-ba74-dfe02ff31224 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:34.765604 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d8e0c-e178-4f75-ba74-dfe02ff31224 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:34.767223 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 455/925] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:34.772474 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213dd3f-a3f1-4891-be8a-35d6e2a0b28b None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:34.773327 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213dd3f-a3f1-4891-be8a-35d6e2a0b28b None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:34.773879 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213dd3f-a3f1-4891-be8a-35d6e2a0b28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:34.793997 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-9213dd3f-a3f1-4891-be8a-35d6e2a0b28b None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:34.795485 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-9213dd3f-a3f1-4891-be8a-35d6e2a0b28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:34.813872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:34.873022 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 471/926] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:13:34.894543 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3d8e0c-e178-4f75-ba74-dfe02ff31224 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:35.004247 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3d8e0c-e178-4f75-ba74-dfe02ff31224 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:35.040860 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0457d-5b69-40d4-9b3b-330599c6a40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:35.041422 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0457d-5b69-40d4-9b3b-330599c6a40f None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:35.041704 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0457d-5b69-40d4-9b3b-330599c6a40f None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:35.041982 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0457d-5b69-40d4-9b3b-330599c6a40f None admin] PATH_INFO: `/v3/services/compute` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:35.042610 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e0457d-5b69-40d4-9b3b-330599c6a40f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:35.043969 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e0457d-5b69-40d4-9b3b-330599c6a40f None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:35.046980 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8231d2f-0b0c-44a1-83dd-744fb2ecbffe None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:35.047265 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8231d2f-0b0c-44a1-83dd-744fb2ecbffe None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:35.047541 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8231d2f-0b0c-44a1-83dd-744fb2ecbffe None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:35.050117 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 472/927] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 15:13:35.053158 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-57e0457d-5b69-40d4-9b3b-330599c6a40f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 15:13:35.053158 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:35.053158 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 15:13:35.053158 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 15:13:35.053158 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 15:13:35.053158 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 15:13:35.053158 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 15:13:35.053158 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:35.053158 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 15:13:35.053158 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:35.053158 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 15:13:35.053158 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:35.053158 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:35.053158 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:35.053158 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:35.053158 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:35.053158 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:35.054154 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:35.054154 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:35.054154 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:35.054154 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:35.054154 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:35.054154 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:35.054154 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 15:13:35.054154 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 15:13:35.054154 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 15:13:35.054154 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 15:13:35.054154 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:35.054154 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:35.054154 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:35.054154 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:35.054154 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:35.054154 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:35.055009 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:35.055009 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:35.055009 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:35.055009 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:35.055009 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:35.055009 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:35.055009 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:35.055009 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:35.055009 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:35.055009 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:35.055009 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 15:13:35.055009 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:35.055009 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 15:13:35.055009 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:35.055768 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93528b1-2854-4d15-a264-34786737068e None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:35.055768 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93528b1-2854-4d15-a264-34786737068e None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:35.055768 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93528b1-2854-4d15-a264-34786737068e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:35.055950 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 456/928] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:13:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:35.060615 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0457d-5b69-40d4-9b3b-330599c6a40f None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:35.081897 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-d93528b1-2854-4d15-a264-34786737068e None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:35.083470 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-d93528b1-2854-4d15-a264-34786737068e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:35.120376 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:35.173220 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0457d-5b69-40d4-9b3b-330599c6a40f None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:35.176208 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 473/929] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 15:13:35.196851 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93528b1-2854-4d15-a264-34786737068e None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:35.223148 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4075a8-fba7-4dab-89eb-85e3a7e552c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:35.224209 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4075a8-fba7-4dab-89eb-85e3a7e552c8 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:35.224958 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4075a8-fba7-4dab-89eb-85e3a7e552c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:35.225459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4075a8-fba7-4dab-89eb-85e3a7e552c8 None admin] PATH_INFO: `/v3/services` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:35.226994 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4075a8-fba7-4dab-89eb-85e3a7e552c8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:35.227493 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4075a8-fba7-4dab-89eb-85e3a7e552c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:35.230137 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4075a8-fba7-4dab-89eb-85e3a7e552c8 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:35.236874 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 457/930] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 5 15:13:35 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:35.244390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4075a8-fba7-4dab-89eb-85e3a7e552c8 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:35.282499 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:35.304129 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93528b1-2854-4d15-a264-34786737068e None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:35.344346 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8e8ae-5160-432c-ae03-32ea8cf61572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:35.345383 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8e8ae-5160-432c-ae03-32ea8cf61572 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:35.345808 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8e8ae-5160-432c-ae03-32ea8cf61572 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:35.346249 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8e8ae-5160-432c-ae03-32ea8cf61572 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:35.357897 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe8e8ae-5160-432c-ae03-32ea8cf61572 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:13:35.363026 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4075a8-fba7-4dab-89eb-85e3a7e552c8 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:35.363529 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-4fe8e8ae-5160-432c-ae03-32ea8cf61572 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 15:13:35.363529 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:35.363529 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:35.363529 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:35.363529 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:35.363529 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:35.363529 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:35.363529 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:35.363529 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:35.363529 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:35.363529 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:35.363529 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:35.363529 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:13:35.363529 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:13:35.363529 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:13:35.363529 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:13:35.364674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:13:35.364674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:13:35.364674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:13:35.364674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:13:35.364674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:35.364674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:35.364674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:35.364674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:35.364674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:35.364674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:35.364674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:35.364674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:35.364674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:35.364674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:35.364674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:35.364674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:35.365710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:35.365710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:35.365710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:35.365710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:35.365710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:13:35.365710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:13:35.365710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:13:35.365710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:13:35.365710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:13:35.365710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:13:35.365710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 15:13:35.365710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:35.368323 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 474/931] 10.222.0.40 () {56 vars in 1120 bytes} [Fri Aug 5 15:13:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:13:35.375373 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8e8ae-5160-432c-ae03-32ea8cf61572 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:35.393852 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:35.416445 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6a18d-0af8-479b-84d4-daefca6b6aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:35.417522 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6a18d-0af8-479b-84d4-daefca6b6aab None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:35.418007 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6a18d-0af8-479b-84d4-daefca6b6aab None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:35.419179 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6a18d-0af8-479b-84d4-daefca6b6aab None admin] PATH_INFO: `/v3/services` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:35.420118 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c6a18d-0af8-479b-84d4-daefca6b6aab None admin] RBAC: Adding query filter params (type=compute) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:35.420644 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c6a18d-0af8-479b-84d4-daefca6b6aab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:35.422754 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c6a18d-0af8-479b-84d4-daefca6b6aab None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:35.436888 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 458/932] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 5 15:13:35 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:35.489460 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8e8ae-5160-432c-ae03-32ea8cf61572 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:35.530687 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5038ffe5-4beb-4dff-9956-6f6753bf0aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:35.531355 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5038ffe5-4beb-4dff-9956-6f6753bf0aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:35.531637 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5038ffe5-4beb-4dff-9956-6f6753bf0aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:35.531912 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5038ffe5-4beb-4dff-9956-6f6753bf0aa7 None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:35.532741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5038ffe5-4beb-4dff-9956-6f6753bf0aa7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:35.533051 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5038ffe5-4beb-4dff-9956-6f6753bf0aa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:35.534623 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5038ffe5-4beb-4dff-9956-6f6753bf0aa7 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:35.537922 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e273d5-3927-471c-a13a-b6e5e3e21960 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:35.538210 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e273d5-3927-471c-a13a-b6e5e3e21960 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:35.538487 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e273d5-3927-471c-a13a-b6e5e3e21960 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:35.541908 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:35.544214 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 459/933] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 15:13:35.546142 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955faea-8810-470c-9c80-f56b7bd9bad0 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:35.546258 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 475/934] 10.222.0.40 () {56 vars in 1115 bytes} [Fri Aug 5 15:13:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:35.547034 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955faea-8810-470c-9c80-f56b7bd9bad0 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:35.556220 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955faea-8810-470c-9c80-f56b7bd9bad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:35.556395 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5038ffe5-4beb-4dff-9956-6f6753bf0aa7 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:35.572434 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-a955faea-8810-470c-9c80-f56b7bd9bad0 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:35.574379 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-a955faea-8810-470c-9c80-f56b7bd9bad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:35.664158 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 460/935] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:13:35.667154 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5038ffe5-4beb-4dff-9956-6f6753bf0aa7 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:35.680048 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955faea-8810-470c-9c80-f56b7bd9bad0 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:35.706427 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc61a3b-ec61-4b18-b329-ca4bd3198061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:35.707685 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc61a3b-ec61-4b18-b329-ca4bd3198061 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:35.708439 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc61a3b-ec61-4b18-b329-ca4bd3198061 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:35.708931 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc61a3b-ec61-4b18-b329-ca4bd3198061 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:35.722686 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc61a3b-ec61-4b18-b329-ca4bd3198061 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:35.724984 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc61a3b-ec61-4b18-b329-ca4bd3198061 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:35.744654 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-afc61a3b-ec61-4b18-b329-ca4bd3198061 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:35.744654 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:35.744654 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:35.744654 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:35.744654 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:35.744654 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:35.744654 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:35.744654 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:35.744654 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:35.744654 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:35.744654 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:35.744654 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:35.744654 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:35.744654 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:35.744654 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:35.745637 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:35.745637 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:35.745637 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:35.745637 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:35.745637 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:35.745637 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:35.745637 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:35.745637 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:35.745637 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:35.745637 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:35.745637 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:35.745637 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:35.745637 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:35.745637 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:35.745637 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:35.745637 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:35.745637 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:35.746610 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:35.746610 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:35.746610 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:35.746610 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:35.749560 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 476/936] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:35.755442 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc61a3b-ec61-4b18-b329-ca4bd3198061 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:35.788046 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955faea-8810-470c-9c80-f56b7bd9bad0 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:35.803192 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:35.817769 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:35.826520 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1241e5f-437c-4349-b3c0-3e6ba9ed2845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:35.827397 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1241e5f-437c-4349-b3c0-3e6ba9ed2845 None admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:35.827933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1241e5f-437c-4349-b3c0-3e6ba9ed2845 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:35.828430 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1241e5f-437c-4349-b3c0-3e6ba9ed2845 None admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:35.829340 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1241e5f-437c-4349-b3c0-3e6ba9ed2845 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:35.831027 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1241e5f-437c-4349-b3c0-3e6ba9ed2845 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:35.833857 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: WARNING keystone.server.flask.common [None req-a1241e5f-437c-4349-b3c0-3e6ba9ed2845 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 05 15:13:35.863594 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc61a3b-ec61-4b18-b329-ca4bd3198061 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:35.877243 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.sql.core [None req-a1241e5f-437c-4349-b3c0-3e6ba9ed2845 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 05 15:13:35.877243 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 05 15:13:35.877243 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [parameters: {'id': 'c144dd49ed814f4b99e848117457ccb3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 05 15:13:35.877243 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80541) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 05 15:13:35.886194 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-a1241e5f-437c-4349-b3c0-3e6ba9ed2845 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c144dd49ed814f4b99e848117457ccb3.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c144dd49ed814f4b99e848117457ccb3. Aug 05 15:13:35.886194 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:35.886194 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 15:13:35.886194 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 15:13:35.886194 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 15:13:35.886194 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 15:13:35.886194 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 15:13:35.886194 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 15:13:35.886194 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 15:13:35.886194 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application conn.query(q) Aug 05 15:13:35.886194 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 15:13:35.886194 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 15:13:35.886194 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 15:13:35.886987 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application result.read() Aug 05 15:13:35.886987 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 15:13:35.886987 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 15:13:35.886987 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 15:13:35.886987 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 15:13:35.886987 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 15:13:35.886987 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 15:13:35.886987 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 15:13:35.886987 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 15:13:35.886987 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 05 15:13:35.886987 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:35.886987 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 05 15:13:35.886987 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:35.886987 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:35.886987 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 05 15:13:35.886987 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 05 15:13:35.886987 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 05 15:13:35.886987 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 05 15:13:35.887915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 15:13:35.887915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application next(self.gen) Aug 05 15:13:35.887915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 05 15:13:35.887915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application yield resource Aug 05 15:13:35.887915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 15:13:35.887915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application next(self.gen) Aug 05 15:13:35.887915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 05 15:13:35.887915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self.session.rollback() Aug 05 15:13:35.887915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 05 15:13:35.887915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self.force_reraise() Aug 05 15:13:35.887915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 05 15:13:35.887915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise self.value Aug 05 15:13:35.887915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 05 15:13:35.887915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 05 15:13:35.887915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 05 15:13:35.887915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application session.commit() Aug 05 15:13:35.887915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 05 15:13:35.888830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 05 15:13:35.888830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 05 15:13:35.888830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self._prepare_impl() Aug 05 15:13:35.888830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 05 15:13:35.888830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self.session.flush() Aug 05 15:13:35.888830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 05 15:13:35.888830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self._flush(objects) Aug 05 15:13:35.888830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 05 15:13:35.888830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 05 15:13:35.888830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 05 15:13:35.888830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application compat.raise_( Aug 05 15:13:35.888830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 15:13:35.888830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception Aug 05 15:13:35.888830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 05 15:13:35.888830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application flush_context.execute() Aug 05 15:13:35.888830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 05 15:13:35.888830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rec.execute(self) Aug 05 15:13:35.889807 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 05 15:13:35.889807 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 05 15:13:35.889807 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 05 15:13:35.889807 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 05 15:13:35.889807 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 05 15:13:35.889807 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 05 15:13:35.889807 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 05 15:13:35.889807 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 05 15:13:35.889807 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 05 15:13:35.889807 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 05 15:13:35.889807 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 05 15:13:35.889807 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 05 15:13:35.889807 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 05 15:13:35.889807 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 05 15:13:35.889807 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 05 15:13:35.892484 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 05 15:13:35.892484 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 15:13:35.892484 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception Aug 05 15:13:35.892484 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 15:13:35.892484 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 15:13:35.892484 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 15:13:35.892484 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 15:13:35.892484 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 15:13:35.892484 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 15:13:35.892484 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 15:13:35.892484 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application conn.query(q) Aug 05 15:13:35.892484 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 15:13:35.892484 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 15:13:35.892484 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 15:13:35.892484 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application result.read() Aug 05 15:13:35.892484 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 15:13:35.892484 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 15:13:35.894589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 15:13:35.894589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 15:13:35.894589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 15:13:35.894589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 15:13:35.894589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 15:13:35.894589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 15:13:35.894589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 05 15:13:35.894589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 05 15:13:35.894589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [parameters: {'id': 'c144dd49ed814f4b99e848117457ccb3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 05 15:13:35.894589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 05 15:13:35.894589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:35.894589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 15:13:35.894589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:35.894589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:35.894589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 05 15:13:35.896671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 05 15:13:35.896671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 05 15:13:35.896671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 05 15:13:35.896671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 05 15:13:35.896671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:35.896671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 15:13:35.896671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:35.896671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:35.896671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:35.896671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:35.896671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:35.896671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:35.896671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:35.896671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:35.896671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:35.896671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:35.896671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:35.900042 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:35.900042 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 05 15:13:35.900042 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 05 15:13:35.900042 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:35.900042 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:35.900042 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 05 15:13:35.900042 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 05 15:13:35.900042 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c144dd49ed814f4b99e848117457ccb3. Aug 05 15:13:35.900042 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:35.900042 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 461/937] 10.222.0.40 () {60 vars in 1140 bytes} [Fri Aug 5 15:13:35 2022] POST /identity/v3/projects => generated 262 bytes in 219 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:13:35.900042 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae5cdde-116b-45cf-aadd-bef31942a176 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:35.905281 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae5cdde-116b-45cf-aadd-bef31942a176 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:35.905281 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae5cdde-116b-45cf-aadd-bef31942a176 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:35.905586 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a23d801-bc86-4c9b-9fde-d23bcb3519eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:35.905586 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23d801-bc86-4c9b-9fde-d23bcb3519eb None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:35.905586 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23d801-bc86-4c9b-9fde-d23bcb3519eb None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:35.905586 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23d801-bc86-4c9b-9fde-d23bcb3519eb None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:35.906935 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a23d801-bc86-4c9b-9fde-d23bcb3519eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:35.906935 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a23d801-bc86-4c9b-9fde-d23bcb3519eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:35.906935 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a23d801-bc86-4c9b-9fde-d23bcb3519eb None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:35.907665 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 462/938] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:35 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:35.909207 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae5cdde-116b-45cf-aadd-bef31942a176 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:35.917905 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 477/939] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:35.923517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496079b-6f74-4307-82b9-f5c3c306c8b8 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:35.923807 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496079b-6f74-4307-82b9-f5c3c306c8b8 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:35.924152 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496079b-6f74-4307-82b9-f5c3c306c8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:35.941571 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-b496079b-6f74-4307-82b9-f5c3c306c8b8 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:35.942766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-b496079b-6f74-4307-82b9-f5c3c306c8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:36.016354 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 478/940] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:13:36.017208 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496079b-6f74-4307-82b9-f5c3c306c8b8 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:36.037222 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae5cdde-116b-45cf-aadd-bef31942a176 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:36.063330 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:36.077667 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890a9a2d-4eac-48b4-aa60-78474071354c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:36.077667 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a9a2d-4eac-48b4-aa60-78474071354c None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:36.077667 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a9a2d-4eac-48b4-aa60-78474071354c None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:36.077667 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a9a2d-4eac-48b4-aa60-78474071354c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:36.083509 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890a9a2d-4eac-48b4-aa60-78474071354c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:36.085168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890a9a2d-4eac-48b4-aa60-78474071354c None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:36.097397 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-890a9a2d-4eac-48b4-aa60-78474071354c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:36.097397 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:36.097397 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:36.097397 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:36.097397 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:36.097397 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:36.097397 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:36.097397 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:36.097397 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:36.097397 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:36.097397 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:36.097397 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:36.097397 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 15:13:36.097397 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 15:13:36.097397 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 15:13:36.098286 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 15:13:36.098286 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:36.098286 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:36.098286 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:36.098286 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:36.098286 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:36.098286 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:36.098286 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:36.098286 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:36.098286 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:36.098286 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:36.098286 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:36.098286 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:36.098286 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:36.098286 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:36.098286 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:36.098286 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:36.099170 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 15:13:36.099170 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 15:13:36.099170 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 15:13:36.099170 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 15:13:36.099170 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 15:13:36.099170 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 15:13:36.099170 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:36.099170 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:36.104113 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 463/941] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:13:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:36.105486 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890a9a2d-4eac-48b4-aa60-78474071354c None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:36.126635 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496079b-6f74-4307-82b9-f5c3c306c8b8 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:36.138261 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:36.164117 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1dbd94-16e3-4b5e-a864-46e3fda9a5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:36.167241 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1dbd94-16e3-4b5e-a864-46e3fda9a5fa None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:36.167241 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1dbd94-16e3-4b5e-a864-46e3fda9a5fa None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:36.167241 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1dbd94-16e3-4b5e-a864-46e3fda9a5fa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:36.179415 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1dbd94-16e3-4b5e-a864-46e3fda9a5fa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:36.180928 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1dbd94-16e3-4b5e-a864-46e3fda9a5fa None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:36.194422 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-eb1dbd94-16e3-4b5e-a864-46e3fda9a5fa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 15:13:36.194422 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:36.194422 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:36.194422 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:36.194422 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:36.194422 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:36.194422 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:36.194422 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:36.194422 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:36.194422 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:36.194422 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:36.194422 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:36.194422 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 15:13:36.194422 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 15:13:36.194422 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 15:13:36.194422 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 15:13:36.195431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:36.195431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:36.195431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 15:13:36.195431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:36.195431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 15:13:36.195431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:36.195431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:36.195431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:36.195431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:36.195431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:36.195431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:36.195431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:36.195431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:36.195431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:36.195431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:36.195431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:36.195431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:36.195431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:36.196407 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:36.196407 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:36.196407 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 15:13:36.196407 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 15:13:36.196407 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 15:13:36.196407 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 15:13:36.196407 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 15:13:36.196407 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 15:13:36.196407 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 15:13:36.196407 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:36.200411 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 479/942] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Aug 5 15:13:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:36.201327 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1dbd94-16e3-4b5e-a864-46e3fda9a5fa None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:36.221404 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890a9a2d-4eac-48b4-aa60-78474071354c None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:36.257492 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afed4f2-005f-4e5f-9d79-c24f4d6ac9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:36.258126 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afed4f2-005f-4e5f-9d79-c24f4d6ac9e0 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:36.258408 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afed4f2-005f-4e5f-9d79-c24f4d6ac9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:36.258685 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afed4f2-005f-4e5f-9d79-c24f4d6ac9e0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:36.259313 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afed4f2-005f-4e5f-9d79-c24f4d6ac9e0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:36.260709 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afed4f2-005f-4e5f-9d79-c24f4d6ac9e0 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:36.269964 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-1afed4f2-005f-4e5f-9d79-c24f4d6ac9e0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 15:13:36.269964 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:36.269964 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 15:13:36.269964 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 15:13:36.269964 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 15:13:36.269964 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 15:13:36.269964 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 15:13:36.269964 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:36.269964 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 15:13:36.269964 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:36.269964 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 15:13:36.269964 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:36.269964 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:36.269964 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:36.269964 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:36.269964 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:36.269964 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:36.270976 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:36.270976 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:36.270976 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:36.270976 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:36.270976 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:36.270976 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:36.270976 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 15:13:36.270976 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 15:13:36.270976 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 15:13:36.270976 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 15:13:36.270976 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:36.270976 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:36.270976 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:36.270976 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:36.270976 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:36.270976 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:36.271837 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:36.271837 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:36.271837 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:36.271837 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:36.271837 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:36.271837 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:36.271837 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:36.271837 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:36.271837 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:36.271837 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:36.271837 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 15:13:36.271837 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:36.271837 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 15:13:36.271837 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:36.271837 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 464/943] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 5 15:13:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:36.272665 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afed4f2-005f-4e5f-9d79-c24f4d6ac9e0 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:36.311587 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1dbd94-16e3-4b5e-a864-46e3fda9a5fa None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:36.313309 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:36.349397 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b70267-097e-45b0-be7e-a4ccc8a19e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:36.349699 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b70267-097e-45b0-be7e-a4ccc8a19e72 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:36.349916 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b70267-097e-45b0-be7e-a4ccc8a19e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:36.350192 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b70267-097e-45b0-be7e-a4ccc8a19e72 None admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:36.350909 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b70267-097e-45b0-be7e-a4ccc8a19e72 None admin] RBAC: Adding query filter params (name=service) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:36.351218 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b70267-097e-45b0-be7e-a4ccc8a19e72 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:36.352687 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b70267-097e-45b0-be7e-a4ccc8a19e72 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:36.370069 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 480/944] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 5 15:13:36 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:36.370688 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b70267-097e-45b0-be7e-a4ccc8a19e72 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:36.381501 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:36.414232 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afed4f2-005f-4e5f-9d79-c24f4d6ac9e0 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:36.464270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df57f848-c2bf-4d63-b2a8-ef607d1fddbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:36.465362 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57f848-c2bf-4d63-b2a8-ef607d1fddbc None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:36.465745 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57f848-c2bf-4d63-b2a8-ef607d1fddbc None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:36.466121 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57f848-c2bf-4d63-b2a8-ef607d1fddbc None admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:36.468957 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df57f848-c2bf-4d63-b2a8-ef607d1fddbc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:36.469445 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df57f848-c2bf-4d63-b2a8-ef607d1fddbc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:36.471005 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df57f848-c2bf-4d63-b2a8-ef607d1fddbc None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:36.491402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 465/945] 10.222.0.40 () {56 vars in 1149 bytes} [Fri Aug 5 15:13:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:36.497408 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df57f848-c2bf-4d63-b2a8-ef607d1fddbc None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:36.502710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b70267-097e-45b0-be7e-a4ccc8a19e72 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:36.547348 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188daa3-e7e7-4c58-9303-53e2d519291b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:36.547981 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188daa3-e7e7-4c58-9303-53e2d519291b None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:36.548302 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188daa3-e7e7-4c58-9303-53e2d519291b None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:36.548580 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188daa3-e7e7-4c58-9303-53e2d519291b None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:36.549274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7188daa3-e7e7-4c58-9303-53e2d519291b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:36.549583 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7188daa3-e7e7-4c58-9303-53e2d519291b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:36.550953 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7188daa3-e7e7-4c58-9303-53e2d519291b None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:36.557181 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 481/946] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Aug 5 15:13:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:36.563732 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188daa3-e7e7-4c58-9303-53e2d519291b None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:36.598428 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:36.619704 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df57f848-c2bf-4d63-b2a8-ef607d1fddbc None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:36.670390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ae3ee-2384-45c7-ae72-44d353936647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:36.670390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ae3ee-2384-45c7-ae72-44d353936647 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:36.670390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ae3ee-2384-45c7-ae72-44d353936647 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:36.670390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ae3ee-2384-45c7-ae72-44d353936647 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:36.685378 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8ae3ee-2384-45c7-ae72-44d353936647 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:36.687824 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8ae3ee-2384-45c7-ae72-44d353936647 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:36.688407 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188daa3-e7e7-4c58-9303-53e2d519291b None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:36.708686 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-5e8ae3ee-2384-45c7-ae72-44d353936647 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:36.708686 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:36.708686 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:36.708686 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:36.708686 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:36.708686 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:36.708686 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:36.708686 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:36.708686 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:36.708686 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:36.708686 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:36.708686 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:36.708686 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:36.708686 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:36.708686 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:36.709806 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:36.709806 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:36.709806 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:36.709806 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:36.709806 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:36.709806 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:36.709806 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:36.709806 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:36.709806 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:36.709806 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:36.709806 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:36.709806 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:36.709806 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:36.709806 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:36.709806 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:36.709806 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:36.709806 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:36.710896 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:36.710896 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:36.710896 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:36.710896 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:36.717517 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 466/947] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:36.719608 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ae3ee-2384-45c7-ae72-44d353936647 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:36.724581 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:36.735742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be2600-a39b-4a80-b82d-8bd165fec75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:36.736857 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be2600-a39b-4a80-b82d-8bd165fec75d None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:36.742517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be2600-a39b-4a80-b82d-8bd165fec75d None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:36.742517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be2600-a39b-4a80-b82d-8bd165fec75d None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:36.742517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46be2600-a39b-4a80-b82d-8bd165fec75d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:36.742517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46be2600-a39b-4a80-b82d-8bd165fec75d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:36.742517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46be2600-a39b-4a80-b82d-8bd165fec75d None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:36.747578 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 482/948] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Aug 5 15:13:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:36.754328 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be2600-a39b-4a80-b82d-8bd165fec75d None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:36.850596 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:36.853252 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ae3ee-2384-45c7-ae72-44d353936647 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:36.860690 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be2600-a39b-4a80-b82d-8bd165fec75d None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:36.897883 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36afb59-ca3f-44c9-9995-7f927e89c44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:36.898489 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36afb59-ca3f-44c9-9995-7f927e89c44e None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:36.898781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36afb59-ca3f-44c9-9995-7f927e89c44e None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:36.899003 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41a063-8c99-4589-b1b1-6859c762ac9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:36.899129 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36afb59-ca3f-44c9-9995-7f927e89c44e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:36.899631 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41a063-8c99-4589-b1b1-6859c762ac9c None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:36.899847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36afb59-ca3f-44c9-9995-7f927e89c44e None admin] RBAC: Adding query filter params (interface=public, service_id=3ecb213366f04902b3464f056d22ee52, region_id=RegionOne) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:36.899951 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41a063-8c99-4589-b1b1-6859c762ac9c None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:36.900206 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36afb59-ca3f-44c9-9995-7f927e89c44e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:36.900309 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41a063-8c99-4589-b1b1-6859c762ac9c None admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:36.900907 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c41a063-8c99-4589-b1b1-6859c762ac9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:36.901214 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c41a063-8c99-4589-b1b1-6859c762ac9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:36.901620 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36afb59-ca3f-44c9-9995-7f927e89c44e None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:36.902551 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c41a063-8c99-4589-b1b1-6859c762ac9c None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:36.910030 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 483/949] 10.222.0.40 () {56 vars in 1251 bytes} [Fri Aug 5 15:13:36 2022] GET /identity/v3/endpoints?service_id=3ecb213366f04902b3464f056d22ee52&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:36.916875 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36afb59-ca3f-44c9-9995-7f927e89c44e None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:36.921014 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 467/950] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:36.926520 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41a063-8c99-4589-b1b1-6859c762ac9c None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:36.991390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:37.040783 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41a063-8c99-4589-b1b1-6859c762ac9c None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:37.053692 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36afb59-ca3f-44c9-9995-7f927e89c44e None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:37.084727 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1166f5f2-d931-47f7-aa4e-1f7b230c7275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:37.085562 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166f5f2-d931-47f7-aa4e-1f7b230c7275 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:37.085939 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166f5f2-d931-47f7-aa4e-1f7b230c7275 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:37.086316 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166f5f2-d931-47f7-aa4e-1f7b230c7275 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:37.093467 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1166f5f2-d931-47f7-aa4e-1f7b230c7275 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:37.094987 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1166f5f2-d931-47f7-aa4e-1f7b230c7275 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:37.098821 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1656e6-d3be-41c8-bab4-8759cce3984b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:37.099618 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1656e6-d3be-41c8-bab4-8759cce3984b None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:37.100050 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1656e6-d3be-41c8-bab4-8759cce3984b None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:37.100534 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1656e6-d3be-41c8-bab4-8759cce3984b None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:37.101471 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1656e6-d3be-41c8-bab4-8759cce3984b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:37.103109 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1656e6-d3be-41c8-bab4-8759cce3984b None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:37.107527 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-1166f5f2-d931-47f7-aa4e-1f7b230c7275 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:37.107527 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:37.107527 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:37.107527 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:37.107527 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:37.107527 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:37.107527 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:37.107527 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:37.107527 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:37.107527 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:37.107527 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:37.107527 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:37.107527 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 15:13:37.107527 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 15:13:37.107527 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 15:13:37.109386 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 15:13:37.109386 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:37.109386 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:37.109386 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:37.109386 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:37.109386 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:37.109386 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:37.109386 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:37.109386 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:37.109386 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:37.109386 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:37.109386 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:37.109386 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:37.109386 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:37.109386 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:37.109386 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:37.109386 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:37.110557 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 15:13:37.110557 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 15:13:37.110557 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 15:13:37.110557 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 15:13:37.110557 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 15:13:37.110557 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 15:13:37.110557 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:37.110557 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:37.116819 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 468/951] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:13:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:13:37.117951 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1166f5f2-d931-47f7-aa4e-1f7b230c7275 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:37.124244 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 484/952] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Aug 5 15:13:36 2022] GET /identity/v3/services => generated 1497 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:13:37.236600 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1166f5f2-d931-47f7-aa4e-1f7b230c7275 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:37.263161 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:37.276172 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e57a518-e46c-499c-97fa-a6311b6d0d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:37.277144 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57a518-e46c-499c-97fa-a6311b6d0d76 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:37.286097 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57a518-e46c-499c-97fa-a6311b6d0d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:37.286097 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57a518-e46c-499c-97fa-a6311b6d0d76 None admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:37.286097 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e57a518-e46c-499c-97fa-a6311b6d0d76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:37.286097 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e57a518-e46c-499c-97fa-a6311b6d0d76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:37.286097 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e57a518-e46c-499c-97fa-a6311b6d0d76 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:37.293960 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 469/953] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 15:13:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:37.300460 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1656e6-d3be-41c8-bab4-8759cce3984b None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:37.324219 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:37.411811 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1656e6-d3be-41c8-bab4-8759cce3984b None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:37.452244 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159cdd6b-23a0-4bc5-a959-6222f89a3b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:37.453444 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159cdd6b-23a0-4bc5-a959-6222f89a3b82 None admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:37.453939 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159cdd6b-23a0-4bc5-a959-6222f89a3b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:37.454425 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159cdd6b-23a0-4bc5-a959-6222f89a3b82 None admin] PATH_INFO: `/v3/projects/06f27ab8886a4c0d81c4d4e3fa100a83/users/1654fb6a81e14d6b99ab495bb6d029d7/roles/e7638910be904dc388b948d044a90758` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:37.456944 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159cdd6b-23a0-4bc5-a959-6222f89a3b82 None admin] RBAC: Authorizing `identity:create_grant(project_id=06f27ab8886a4c0d81c4d4e3fa100a83, user_id=1654fb6a81e14d6b99ab495bb6d029d7, role_id=e7638910be904dc388b948d044a90758)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:37.458596 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159cdd6b-23a0-4bc5-a959-6222f89a3b82 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:37.545930 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 485/954] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 15:13:37 2022] PUT /identity/v3/projects/06f27ab8886a4c0d81c4d4e3fa100a83/users/1654fb6a81e14d6b99ab495bb6d029d7/roles/e7638910be904dc388b948d044a90758 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:13:37.575515 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f56f9f-6597-43fc-9e8f-96bc57b63554 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:37.576187 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f56f9f-6597-43fc-9e8f-96bc57b63554 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:37.576665 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f56f9f-6597-43fc-9e8f-96bc57b63554 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:37.578716 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 470/955] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:37.584801 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4501455-1e71-426c-8a63-363071b5274e None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:37.585394 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4501455-1e71-426c-8a63-363071b5274e None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:37.585845 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4501455-1e71-426c-8a63-363071b5274e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:37.606989 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-f4501455-1e71-426c-8a63-363071b5274e None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:37.608829 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-f4501455-1e71-426c-8a63-363071b5274e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:37.690718 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 486/956] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:13:37.714325 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f56f9f-6597-43fc-9e8f-96bc57b63554 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:37.825470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f56f9f-6597-43fc-9e8f-96bc57b63554 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:37.860898 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b968ba-681b-4db9-8daf-b2b4e300e2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:37.861520 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b968ba-681b-4db9-8daf-b2b4e300e2e4 None admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:37.861781 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b968ba-681b-4db9-8daf-b2b4e300e2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:37.862056 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b968ba-681b-4db9-8daf-b2b4e300e2e4 None admin] PATH_INFO: `/v3/services` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:37.862754 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b968ba-681b-4db9-8daf-b2b4e300e2e4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:37.864153 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b968ba-681b-4db9-8daf-b2b4e300e2e4 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:37.953323 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 471/957] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 5 15:13:37 2022] POST /identity/v3/services => generated 242 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:13:38.565357 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58ab50-7a2d-47f2-b253-1922201a8ae7 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:38.565909 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58ab50-7a2d-47f2-b253-1922201a8ae7 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:38.565909 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58ab50-7a2d-47f2-b253-1922201a8ae7 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:38.567597 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 487/958] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:38.572907 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6dfa5-6f2d-4f55-81d3-4dd52f9b433a None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:38.573200 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6dfa5-6f2d-4f55-81d3-4dd52f9b433a None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:38.573481 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6dfa5-6f2d-4f55-81d3-4dd52f9b433a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:38.591874 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-e5d6dfa5-6f2d-4f55-81d3-4dd52f9b433a None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:38.593470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-e5d6dfa5-6f2d-4f55-81d3-4dd52f9b433a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:38.629949 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:38.668572 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 472/959] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:38 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:13:38.694294 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58ab50-7a2d-47f2-b253-1922201a8ae7 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:38.705620 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:38.803379 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58ab50-7a2d-47f2-b253-1922201a8ae7 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:38.839528 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f8cc9d-4bb2-48c9-be34-4232b229b5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:38.840083 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f8cc9d-4bb2-48c9-be34-4232b229b5da None admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:38.840372 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f8cc9d-4bb2-48c9-be34-4232b229b5da None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:38.840650 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f8cc9d-4bb2-48c9-be34-4232b229b5da None admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:38.841323 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f8cc9d-4bb2-48c9-be34-4232b229b5da None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:38.842757 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f8cc9d-4bb2-48c9-be34-4232b229b5da None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:38.845334 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: WARNING keystone.server.flask.common [None req-95f8cc9d-4bb2-48c9-be34-4232b229b5da None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 05 15:13:38.887256 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.sql.core [None req-95f8cc9d-4bb2-48c9-be34-4232b229b5da None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 05 15:13:38.887256 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 05 15:13:38.887256 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [parameters: {'user_id': '8f40c0dabc574f17ab7399145f5e93f0', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 05 15:13:38.887256 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80542) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 05 15:13:38.903822 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-95f8cc9d-4bb2-48c9-be34-4232b229b5da None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 05 15:13:38.903822 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:38.903822 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 15:13:38.903822 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 15:13:38.903822 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 15:13:38.903822 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 15:13:38.903822 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 15:13:38.903822 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 15:13:38.903822 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 15:13:38.903822 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application conn.query(q) Aug 05 15:13:38.903822 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 15:13:38.903822 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 15:13:38.903822 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 15:13:38.903822 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application result.read() Aug 05 15:13:38.903822 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 15:13:38.904766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 15:13:38.904766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 15:13:38.904766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 15:13:38.904766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 15:13:38.904766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 15:13:38.904766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 15:13:38.904766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 15:13:38.904766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 05 15:13:38.904766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:38.904766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 05 15:13:38.904766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:38.904766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:38.904766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 05 15:13:38.904766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 05 15:13:38.904766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 05 15:13:38.904766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 05 15:13:38.904766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 15:13:38.904766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application next(self.gen) Aug 05 15:13:38.906869 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 05 15:13:38.906869 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application yield resource Aug 05 15:13:38.906869 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 15:13:38.906869 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application next(self.gen) Aug 05 15:13:38.906869 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 05 15:13:38.906869 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application self.session.rollback() Aug 05 15:13:38.906869 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 05 15:13:38.906869 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application self.force_reraise() Aug 05 15:13:38.906869 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 05 15:13:38.906869 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise self.value Aug 05 15:13:38.906869 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 05 15:13:38.906869 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 05 15:13:38.906869 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 05 15:13:38.906869 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application session.commit() Aug 05 15:13:38.906869 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 05 15:13:38.906869 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 05 15:13:38.906869 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 05 15:13:38.907781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application self._prepare_impl() Aug 05 15:13:38.907781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 05 15:13:38.907781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application self.session.flush() Aug 05 15:13:38.907781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 05 15:13:38.907781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application self._flush(objects) Aug 05 15:13:38.907781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 05 15:13:38.907781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 05 15:13:38.907781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 05 15:13:38.907781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application compat.raise_( Aug 05 15:13:38.907781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 15:13:38.907781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception Aug 05 15:13:38.907781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 05 15:13:38.907781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application flush_context.execute() Aug 05 15:13:38.907781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 05 15:13:38.907781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rec.execute(self) Aug 05 15:13:38.907781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 05 15:13:38.907781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 05 15:13:38.908698 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 05 15:13:38.908698 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 05 15:13:38.908698 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 05 15:13:38.908698 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 05 15:13:38.908698 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 05 15:13:38.908698 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 05 15:13:38.908698 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 05 15:13:38.908698 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 05 15:13:38.908698 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 05 15:13:38.908698 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 05 15:13:38.908698 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 05 15:13:38.908698 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 05 15:13:38.908698 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 05 15:13:38.908698 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 05 15:13:38.908698 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 15:13:38.908698 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception Aug 05 15:13:38.909535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 15:13:38.909535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 15:13:38.909535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 15:13:38.909535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 15:13:38.909535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 15:13:38.909535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 15:13:38.909535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 15:13:38.909535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application conn.query(q) Aug 05 15:13:38.909535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 15:13:38.909535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 15:13:38.909535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 15:13:38.909535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application result.read() Aug 05 15:13:38.909535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 15:13:38.909535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 15:13:38.909535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 15:13:38.909535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 15:13:38.911530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 15:13:38.911530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 15:13:38.911530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 15:13:38.911530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 15:13:38.911530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 05 15:13:38.911530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 05 15:13:38.911530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [parameters: {'user_id': '8f40c0dabc574f17ab7399145f5e93f0', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 05 15:13:38.911530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 05 15:13:38.911530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:38.911530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 15:13:38.911530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:38.911530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:38.911530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:38.911530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:38.911530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:38.917575 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:38.917575 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:38.917575 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:38.917575 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:38.917575 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:38.917575 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:38.917575 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:38.917575 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 05 15:13:38.917575 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 05 15:13:38.917575 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:38.917575 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:38.917575 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 15:13:38.917575 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:38.917575 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 15:13:38.917575 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:38.917575 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 05 15:13:38.917575 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 05 15:13:38.919631 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 05 15:13:38.919631 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 05 15:13:38.919631 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 05 15:13:38.919631 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 05 15:13:38.919631 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 05 15:13:38.919631 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:38.919631 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 488/960] 10.222.0.40 () {60 vars in 1131 bytes} [Fri Aug 5 15:13:38 2022] POST /identity/v3/users => generated 160 bytes in 211 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:38.920130 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6dfa5-6f2d-4f55-81d3-4dd52f9b433a None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:39.014751 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6dfa5-6f2d-4f55-81d3-4dd52f9b433a None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:39.050605 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f3d5d-2b6c-4750-b94d-6ce46dc4aaa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:39.051259 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f3d5d-2b6c-4750-b94d-6ce46dc4aaa9 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:39.051547 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f3d5d-2b6c-4750-b94d-6ce46dc4aaa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:39.051828 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f3d5d-2b6c-4750-b94d-6ce46dc4aaa9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:39.059378 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2f3d5d-2b6c-4750-b94d-6ce46dc4aaa9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:39.060853 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2f3d5d-2b6c-4750-b94d-6ce46dc4aaa9 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:39.072854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-de2f3d5d-2b6c-4750-b94d-6ce46dc4aaa9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 15:13:39.072854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:39.072854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:39.072854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:39.072854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:39.072854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:39.072854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:39.072854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:39.072854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:39.072854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:39.072854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:39.072854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:39.072854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 15:13:39.072854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 15:13:39.072854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 15:13:39.072854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 15:13:39.074952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:39.074952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:39.074952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 15:13:39.074952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:39.074952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 15:13:39.074952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:39.074952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:39.074952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:39.074952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:39.074952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:39.074952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:39.074952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:39.074952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:39.074952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:39.074952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:39.074952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:39.074952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:39.074952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:39.076178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:39.076178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:39.076178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 15:13:39.076178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 15:13:39.076178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 15:13:39.076178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 15:13:39.076178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 15:13:39.076178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 15:13:39.076178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 15:13:39.076178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:39.076178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 473/961] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:13:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:39.081350 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f8cc9d-4bb2-48c9-be34-4232b229b5da None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:39.158290 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:39.190007 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f8cc9d-4bb2-48c9-be34-4232b229b5da None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:39.225646 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd6b0ff-ed1a-45ff-92cb-f13f0dd69c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:39.226272 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd6b0ff-ed1a-45ff-92cb-f13f0dd69c55 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:39.226555 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd6b0ff-ed1a-45ff-92cb-f13f0dd69c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:39.226831 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd6b0ff-ed1a-45ff-92cb-f13f0dd69c55 None admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:39.227601 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd6b0ff-ed1a-45ff-92cb-f13f0dd69c55 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:39.227909 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd6b0ff-ed1a-45ff-92cb-f13f0dd69c55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:39.229585 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd6b0ff-ed1a-45ff-92cb-f13f0dd69c55 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:39.269674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 489/962] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 5 15:13:39 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:39.293647 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07443640-bd79-4387-a2a6-1015b56e8186 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:39.293997 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07443640-bd79-4387-a2a6-1015b56e8186 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:39.294223 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07443640-bd79-4387-a2a6-1015b56e8186 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:39.295999 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 474/963] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:39.302646 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b2d0d-3aab-417e-a1c8-80be258e505c None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:39.302961 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b2d0d-3aab-417e-a1c8-80be258e505c None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:39.303216 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b2d0d-3aab-417e-a1c8-80be258e505c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:39.323197 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-d31b2d0d-3aab-417e-a1c8-80be258e505c None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:39.324839 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-d31b2d0d-3aab-417e-a1c8-80be258e505c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:39.401973 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 490/964] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:39 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:13:39.426257 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07443640-bd79-4387-a2a6-1015b56e8186 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:39.462807 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:39.544995 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07443640-bd79-4387-a2a6-1015b56e8186 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:39.579996 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce4abe6-9e7e-4edc-b580-6e992b628fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:39.580678 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4abe6-9e7e-4edc-b580-6e992b628fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:39.580967 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4abe6-9e7e-4edc-b580-6e992b628fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:39.581246 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4abe6-9e7e-4edc-b580-6e992b628fa0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:39.581869 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce4abe6-9e7e-4edc-b580-6e992b628fa0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:39.583304 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce4abe6-9e7e-4edc-b580-6e992b628fa0 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:39.592213 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-5ce4abe6-9e7e-4edc-b580-6e992b628fa0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 15:13:39.592213 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:39.592213 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 15:13:39.592213 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 15:13:39.592213 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 15:13:39.592213 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 15:13:39.592213 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 15:13:39.592213 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:39.592213 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 15:13:39.592213 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:39.592213 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 15:13:39.592213 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:39.592213 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:39.592213 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:39.592213 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:39.592213 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:39.592213 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:39.593854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:39.593854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:39.593854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:39.593854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:39.593854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:39.593854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:39.593854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 15:13:39.593854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 15:13:39.593854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 15:13:39.593854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 15:13:39.593854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:39.593854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:39.593854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:39.593854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:39.593854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:39.593854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:39.594727 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:39.594727 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:39.594727 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:39.594727 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:39.594727 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:39.594727 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:39.594727 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:39.594727 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:39.594727 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:39.594727 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:39.594727 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 15:13:39.594727 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:39.594727 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 15:13:39.594727 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:39.594727 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 475/965] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 5 15:13:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:39.601597 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31b2d0d-3aab-417e-a1c8-80be258e505c None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:39.661069 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:39.703467 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01adb56b-9579-499d-af94-28083954c182 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:39.704004 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31b2d0d-3aab-417e-a1c8-80be258e505c None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:39.704298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01adb56b-9579-499d-af94-28083954c182 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:39.704594 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01adb56b-9579-499d-af94-28083954c182 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:39.706093 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 476/966] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 15:13:39.710806 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072439f2-49c8-47c7-8fac-37f49ab887d0 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:39.711093 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072439f2-49c8-47c7-8fac-37f49ab887d0 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:39.711369 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072439f2-49c8-47c7-8fac-37f49ab887d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:39.729243 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-072439f2-49c8-47c7-8fac-37f49ab887d0 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:39.730432 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-072439f2-49c8-47c7-8fac-37f49ab887d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:39.739336 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be2c397-664d-44d3-80db-cdb2b39c1e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:39.739959 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be2c397-664d-44d3-80db-cdb2b39c1e34 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:39.740277 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be2c397-664d-44d3-80db-cdb2b39c1e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:39.740557 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be2c397-664d-44d3-80db-cdb2b39c1e34 None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:39.741235 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be2c397-664d-44d3-80db-cdb2b39c1e34 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:39.741614 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be2c397-664d-44d3-80db-cdb2b39c1e34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:39.742922 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be2c397-664d-44d3-80db-cdb2b39c1e34 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:39.748617 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 491/967] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Aug 5 15:13:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:39.754242 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be2c397-664d-44d3-80db-cdb2b39c1e34 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:39.807271 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 477/968] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:39 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 15:13:39.831209 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072439f2-49c8-47c7-8fac-37f49ab887d0 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:39.855626 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:39.860173 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be2c397-664d-44d3-80db-cdb2b39c1e34 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:39.885419 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:39.898278 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c192a67-a680-49f2-8a69-59d268b141f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:39.899005 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c192a67-a680-49f2-8a69-59d268b141f8 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:39.899211 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c192a67-a680-49f2-8a69-59d268b141f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:39.899457 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c192a67-a680-49f2-8a69-59d268b141f8 None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:39.900166 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c192a67-a680-49f2-8a69-59d268b141f8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:39.900479 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c192a67-a680-49f2-8a69-59d268b141f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:39.901844 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c192a67-a680-49f2-8a69-59d268b141f8 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:39.911530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 492/969] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Aug 5 15:13:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:39.912548 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c192a67-a680-49f2-8a69-59d268b141f8 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:39.938441 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072439f2-49c8-47c7-8fac-37f49ab887d0 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:39.979427 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf3639-6053-46cc-bb50-edb26f49f2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:39.979971 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf3639-6053-46cc-bb50-edb26f49f2b8 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:39.980285 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf3639-6053-46cc-bb50-edb26f49f2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:39.980559 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf3639-6053-46cc-bb50-edb26f49f2b8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:39.993337 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcf3639-6053-46cc-bb50-edb26f49f2b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:13:40.000279 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-cfcf3639-6053-46cc-bb50-edb26f49f2b8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 15:13:40.000279 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:40.000279 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:40.000279 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:40.000279 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:40.000279 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:40.000279 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:40.000279 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:40.000279 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:40.000279 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:40.000279 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:40.000279 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:40.000279 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:13:40.000279 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:13:40.000279 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:13:40.000279 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:13:40.001444 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:13:40.001444 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:13:40.001444 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:13:40.001444 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:13:40.001444 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:40.001444 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:40.001444 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:40.001444 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:40.001444 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:40.001444 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:40.001444 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:40.001444 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:40.001444 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:40.001444 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:40.001444 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:40.001444 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:40.002298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:40.002298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:40.002298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:40.002298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:40.002298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:13:40.002298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:13:40.002298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:13:40.002298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:13:40.002298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:13:40.002298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:13:40.002298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 15:13:40.002298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:40.002298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 478/970] 10.222.0.40 () {56 vars in 1120 bytes} [Fri Aug 5 15:13:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:13:40.007411 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf3639-6053-46cc-bb50-edb26f49f2b8 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:40.026336 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c192a67-a680-49f2-8a69-59d268b141f8 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:40.063990 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c6b38-7ca3-4009-81a8-fd75f53a487e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:40.064628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c6b38-7ca3-4009-81a8-fd75f53a487e None admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:40.064909 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c6b38-7ca3-4009-81a8-fd75f53a487e None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:40.065180 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c6b38-7ca3-4009-81a8-fd75f53a487e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:40.065905 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c6b38-7ca3-4009-81a8-fd75f53a487e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:40.067258 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c6b38-7ca3-4009-81a8-fd75f53a487e None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:40.091530 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:40.109203 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf3639-6053-46cc-bb50-edb26f49f2b8 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:40.132647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 493/971] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:13:39 2022] POST /identity/v3/endpoints => generated 349 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:13:40.133968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3226c-6c7c-4c3b-ac0c-51ef0367fa2b None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:40.134263 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3226c-6c7c-4c3b-ac0c-51ef0367fa2b None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:40.134539 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3226c-6c7c-4c3b-ac0c-51ef0367fa2b None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:40.135762 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 494/972] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:40.141394 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ba0cd-b456-4d74-bed9-0961d324334f None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:40.141692 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ba0cd-b456-4d74-bed9-0961d324334f None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:40.141932 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ba0cd-b456-4d74-bed9-0961d324334f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:40.162457 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-7d0ba0cd-b456-4d74-bed9-0961d324334f None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:40.166348 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-7d0ba0cd-b456-4d74-bed9-0961d324334f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:40.167282 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad94a2e-4745-4ba0-a16c-542d18f5d6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:40.168055 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad94a2e-4745-4ba0-a16c-542d18f5d6df None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:40.168370 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad94a2e-4745-4ba0-a16c-542d18f5d6df None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:40.168651 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad94a2e-4745-4ba0-a16c-542d18f5d6df None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:40.169439 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad94a2e-4745-4ba0-a16c-542d18f5d6df None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:40.172531 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad94a2e-4745-4ba0-a16c-542d18f5d6df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:40.174084 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad94a2e-4745-4ba0-a16c-542d18f5d6df None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:40.190151 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 479/973] 10.222.0.40 () {56 vars in 1115 bytes} [Fri Aug 5 15:13:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:40.191640 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad94a2e-4745-4ba0-a16c-542d18f5d6df None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:40.226870 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:40.265271 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 495/974] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:40 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 15:13:40.289627 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ba0cd-b456-4d74-bed9-0961d324334f None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:40.317155 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad94a2e-4745-4ba0-a16c-542d18f5d6df None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:40.356507 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0fc8f0-415a-46ce-bd77-225fa41f6570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:40.357148 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0fc8f0-415a-46ce-bd77-225fa41f6570 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:40.357431 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0fc8f0-415a-46ce-bd77-225fa41f6570 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:40.357707 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0fc8f0-415a-46ce-bd77-225fa41f6570 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:40.367908 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:40.370832 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0fc8f0-415a-46ce-bd77-225fa41f6570 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:40.372689 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0fc8f0-415a-46ce-bd77-225fa41f6570 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:40.388008 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-1c0fc8f0-415a-46ce-bd77-225fa41f6570 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:40.388008 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:40.388008 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:40.388008 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:40.388008 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:40.388008 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:40.388008 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:40.388008 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:40.388008 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:40.388008 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:40.388008 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:40.388008 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:40.388008 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:40.388008 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:40.388008 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:40.389259 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:40.389259 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:40.389259 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:40.389259 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:40.389259 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:40.389259 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:40.389259 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:40.389259 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:40.389259 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:40.389259 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:40.389259 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:40.389259 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:40.389259 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:40.389259 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:40.389259 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:40.389259 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:40.389259 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:40.390274 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:40.390274 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:40.390274 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:40.390274 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:40.390274 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 480/975] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:13:40.395021 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0fc8f0-415a-46ce-bd77-225fa41f6570 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:40.398039 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ba0cd-b456-4d74-bed9-0961d324334f None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:40.434183 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae43f367-9675-4363-b282-440f6beca56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:40.434828 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43f367-9675-4363-b282-440f6beca56f None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:40.435117 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43f367-9675-4363-b282-440f6beca56f None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:40.435392 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43f367-9675-4363-b282-440f6beca56f None admin] PATH_INFO: `/v3/services/compute` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:40.436020 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae43f367-9675-4363-b282-440f6beca56f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:40.437518 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae43f367-9675-4363-b282-440f6beca56f None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:40.446508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-ae43f367-9675-4363-b282-440f6beca56f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 15:13:40.446508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:40.446508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 15:13:40.446508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 15:13:40.446508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 15:13:40.446508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 15:13:40.446508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 15:13:40.446508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:40.446508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 15:13:40.446508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:40.446508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 15:13:40.446508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:40.446508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:40.446508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:40.446508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:40.446508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:40.446508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:40.447527 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:40.447527 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:40.447527 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:40.447527 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:40.447527 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:40.447527 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:40.447527 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 15:13:40.447527 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 15:13:40.447527 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 15:13:40.447527 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 15:13:40.447527 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:40.447527 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:40.447527 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:40.447527 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:40.447527 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:40.447527 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:40.448402 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:40.448402 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:40.448402 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:40.448402 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:40.448402 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:40.448402 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:40.448402 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:40.448402 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:40.448402 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:40.448402 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:40.448402 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 15:13:40.448402 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:40.448402 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 15:13:40.448402 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:40.448402 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 496/976] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:13:40 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:13:40.453577 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae43f367-9675-4363-b282-440f6beca56f None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:40.492802 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:40.510771 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0fc8f0-415a-46ce-bd77-225fa41f6570 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:40.555147 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b68bc95-1acf-4c68-9689-ccaa3c842812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:40.555702 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68bc95-1acf-4c68-9689-ccaa3c842812 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:40.556529 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68bc95-1acf-4c68-9689-ccaa3c842812 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:40.556758 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68bc95-1acf-4c68-9689-ccaa3c842812 None admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:40.557450 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b68bc95-1acf-4c68-9689-ccaa3c842812 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:40.557764 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b68bc95-1acf-4c68-9689-ccaa3c842812 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:40.559110 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b68bc95-1acf-4c68-9689-ccaa3c842812 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:40.563150 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae43f367-9675-4363-b282-440f6beca56f None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:40.571724 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 481/977] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:40.577087 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b68bc95-1acf-4c68-9689-ccaa3c842812 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:40.604314 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721c2c9-2923-413b-8148-a7cd92792b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:40.605030 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721c2c9-2923-413b-8148-a7cd92792b97 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:40.605387 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721c2c9-2923-413b-8148-a7cd92792b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:40.605736 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721c2c9-2923-413b-8148-a7cd92792b97 None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:40.606492 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d721c2c9-2923-413b-8148-a7cd92792b97 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:40.606891 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d721c2c9-2923-413b-8148-a7cd92792b97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:40.608360 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d721c2c9-2923-413b-8148-a7cd92792b97 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:40.615839 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 497/978] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 5 15:13:40 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:40.621670 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721c2c9-2923-413b-8148-a7cd92792b97 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:40.633898 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:40.686149 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b68bc95-1acf-4c68-9689-ccaa3c842812 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:40.721542 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cbb555-d7f1-4d5f-82f7-a61f5ab223d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:40.722264 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cbb555-d7f1-4d5f-82f7-a61f5ab223d9 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:40.722498 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:40.722671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cbb555-d7f1-4d5f-82f7-a61f5ab223d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:40.723047 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cbb555-d7f1-4d5f-82f7-a61f5ab223d9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:40.726475 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721c2c9-2923-413b-8148-a7cd92792b97 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:40.730469 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cbb555-d7f1-4d5f-82f7-a61f5ab223d9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:40.731900 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cbb555-d7f1-4d5f-82f7-a61f5ab223d9 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:40.744493 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-c4cbb555-d7f1-4d5f-82f7-a61f5ab223d9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 15:13:40.744493 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:40.744493 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:40.744493 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:40.744493 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:40.744493 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:40.744493 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:40.744493 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:40.744493 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:40.744493 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:40.744493 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:40.744493 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:40.744493 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 15:13:40.744493 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 15:13:40.744493 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 15:13:40.744493 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 15:13:40.745463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:40.745463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:40.745463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 15:13:40.745463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:40.745463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 15:13:40.745463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:40.745463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:40.745463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:40.745463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:40.745463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:40.745463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:40.745463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:40.745463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:40.745463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:40.745463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:40.745463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:40.745463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:40.745463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:40.746424 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:40.746424 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:40.746424 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 15:13:40.746424 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 15:13:40.746424 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 15:13:40.746424 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 15:13:40.746424 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 15:13:40.746424 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 15:13:40.746424 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 15:13:40.746424 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:40.750613 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 482/979] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Aug 5 15:13:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:13:40.751626 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cbb555-d7f1-4d5f-82f7-a61f5ab223d9 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:40.763387 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba7adf-5ca3-4e27-9748-486ae8752cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:40.764023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba7adf-5ca3-4e27-9748-486ae8752cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:40.764338 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba7adf-5ca3-4e27-9748-486ae8752cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:40.764619 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba7adf-5ca3-4e27-9748-486ae8752cc7 None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:40.765336 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ba7adf-5ca3-4e27-9748-486ae8752cc7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:40.765650 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ba7adf-5ca3-4e27-9748-486ae8752cc7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:40.767020 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ba7adf-5ca3-4e27-9748-486ae8752cc7 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:40.776533 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 498/980] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 5 15:13:40 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:40.777569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba7adf-5ca3-4e27-9748-486ae8752cc7 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:40.857899 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cbb555-d7f1-4d5f-82f7-a61f5ab223d9 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:40.871669 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:40.884840 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba7adf-5ca3-4e27-9748-486ae8752cc7 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:40.896099 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27d694-0927-4022-8099-1ef6da0d10a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:40.896852 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27d694-0927-4022-8099-1ef6da0d10a8 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:40.897221 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27d694-0927-4022-8099-1ef6da0d10a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:40.897596 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27d694-0927-4022-8099-1ef6da0d10a8 None admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:40.898608 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb27d694-0927-4022-8099-1ef6da0d10a8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:40.899105 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb27d694-0927-4022-8099-1ef6da0d10a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:40.900641 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb27d694-0927-4022-8099-1ef6da0d10a8 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:40.920170 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee051e46-b73a-4d7a-8374-1058a52a7015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:40.920794 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee051e46-b73a-4d7a-8374-1058a52a7015 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:40.921116 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee051e46-b73a-4d7a-8374-1058a52a7015 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:40.921399 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee051e46-b73a-4d7a-8374-1058a52a7015 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:40.922164 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee051e46-b73a-4d7a-8374-1058a52a7015 None admin] RBAC: Adding query filter params (interface=public, service_id=0ade76b975a94521b88de5691e0c8833, region_id=RegionOne) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:40.922483 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee051e46-b73a-4d7a-8374-1058a52a7015 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:40.923850 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee051e46-b73a-4d7a-8374-1058a52a7015 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:40.924219 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 483/981] 10.222.0.40 () {56 vars in 1149 bytes} [Fri Aug 5 15:13:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:40.925431 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27d694-0927-4022-8099-1ef6da0d10a8 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:40.933948 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 499/982] 10.222.0.40 () {56 vars in 1251 bytes} [Fri Aug 5 15:13:40 2022] GET /identity/v3/endpoints?service_id=0ade76b975a94521b88de5691e0c8833&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:40.935777 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee051e46-b73a-4d7a-8374-1058a52a7015 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:40.966217 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:41.035471 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27d694-0927-4022-8099-1ef6da0d10a8 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:41.039330 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee051e46-b73a-4d7a-8374-1058a52a7015 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:41.070552 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce17d0-7932-44f6-bd65-b6dfd4884956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:41.071166 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce17d0-7932-44f6-bd65-b6dfd4884956 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:41.071544 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce17d0-7932-44f6-bd65-b6dfd4884956 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:41.071931 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce17d0-7932-44f6-bd65-b6dfd4884956 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:41.073032 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b1e70e-b586-4290-8698-5dc069a6b8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:41.073650 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1e70e-b586-4290-8698-5dc069a6b8cb None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:41.073939 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1e70e-b586-4290-8698-5dc069a6b8cb None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:41.074221 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1e70e-b586-4290-8698-5dc069a6b8cb None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:41.075052 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b1e70e-b586-4290-8698-5dc069a6b8cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:41.077744 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b1e70e-b586-4290-8698-5dc069a6b8cb None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:41.096098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 500/983] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Aug 5 15:13:40 2022] GET /identity/v3/services => generated 1727 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 15:13:41.126593 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:41.158208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ce17d0-7932-44f6-bd65-b6dfd4884956 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:41.159681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ce17d0-7932-44f6-bd65-b6dfd4884956 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:41.276297 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 484/984] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:13:40 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 05 15:13:41.282106 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b1e70e-b586-4290-8698-5dc069a6b8cb None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:41.375684 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:41.384490 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b1e70e-b586-4290-8698-5dc069a6b8cb None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:41.418099 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d80af54-dffe-44ff-8218-4d0bcea382aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:41.418738 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d80af54-dffe-44ff-8218-4d0bcea382aa None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:41.419019 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d80af54-dffe-44ff-8218-4d0bcea382aa None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:41.419295 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d80af54-dffe-44ff-8218-4d0bcea382aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:41.429848 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d80af54-dffe-44ff-8218-4d0bcea382aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:41.431277 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d80af54-dffe-44ff-8218-4d0bcea382aa None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:41.445943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-9d80af54-dffe-44ff-8218-4d0bcea382aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:41.445943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:41.445943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:41.445943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:41.445943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:41.445943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:41.445943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:41.445943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:41.445943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:41.445943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:41.445943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:41.445943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:41.445943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:41.445943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:41.445943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:41.446850 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:41.446850 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:41.446850 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:41.446850 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:41.446850 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:41.446850 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:41.446850 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:41.446850 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:41.446850 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:41.446850 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:41.446850 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:41.446850 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:41.446850 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:41.446850 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:41.446850 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:41.446850 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:41.446850 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:41.447854 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:41.447854 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:41.447854 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:41.447854 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:41.451801 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 501/985] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:41.453183 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce17d0-7932-44f6-bd65-b6dfd4884956 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:41.521978 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:41.554280 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce17d0-7932-44f6-bd65-b6dfd4884956 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:41.589241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee59460-1d07-4c0b-bb23-4bfc0a3ba397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:41.589857 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee59460-1d07-4c0b-bb23-4bfc0a3ba397 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:41.590139 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee59460-1d07-4c0b-bb23-4bfc0a3ba397 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:41.590412 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee59460-1d07-4c0b-bb23-4bfc0a3ba397 None admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:41.591095 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee59460-1d07-4c0b-bb23-4bfc0a3ba397 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:41.591404 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee59460-1d07-4c0b-bb23-4bfc0a3ba397 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:41.592808 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee59460-1d07-4c0b-bb23-4bfc0a3ba397 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:41.609092 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 485/986] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:41.610934 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d80af54-dffe-44ff-8218-4d0bcea382aa None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:41.715693 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d80af54-dffe-44ff-8218-4d0bcea382aa None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:41.751417 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd7c7dc-7c3a-4537-9986-a89e1b2f5b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:41.752096 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7c7dc-7c3a-4537-9986-a89e1b2f5b97 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:41.752385 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7c7dc-7c3a-4537-9986-a89e1b2f5b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:41.752666 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7c7dc-7c3a-4537-9986-a89e1b2f5b97 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:41.759535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd7c7dc-7c3a-4537-9986-a89e1b2f5b97 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:41.761024 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd7c7dc-7c3a-4537-9986-a89e1b2f5b97 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:41.773060 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-6bd7c7dc-7c3a-4537-9986-a89e1b2f5b97 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:41.773060 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:41.773060 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:41.773060 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:41.773060 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:41.773060 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:41.773060 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:41.773060 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:41.773060 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:41.773060 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:41.773060 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:41.773060 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:41.773060 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 15:13:41.773060 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 15:13:41.773060 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 15:13:41.773979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 15:13:41.773979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:41.773979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:41.773979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:41.773979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:41.773979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:41.773979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:41.773979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:41.773979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:41.773979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:41.773979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:41.773979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:41.773979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:41.773979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:41.773979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:41.773979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:41.773979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:41.774961 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 15:13:41.774961 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 15:13:41.774961 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 15:13:41.774961 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 15:13:41.774961 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 15:13:41.774961 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 15:13:41.774961 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:41.774961 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:41.776378 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 502/987] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:13:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:41.780092 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee59460-1d07-4c0b-bb23-4bfc0a3ba397 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:41.885668 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee59460-1d07-4c0b-bb23-4bfc0a3ba397 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:41.914320 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:41.919803 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9ce837-7f14-4f9c-a378-d2ca57cd49d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:41.920461 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ce837-7f14-4f9c-a378-d2ca57cd49d5 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:41.920752 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ce837-7f14-4f9c-a378-d2ca57cd49d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:41.921590 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ce837-7f14-4f9c-a378-d2ca57cd49d5 None admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:41.922338 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9ce837-7f14-4f9c-a378-d2ca57cd49d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:41.922651 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9ce837-7f14-4f9c-a378-d2ca57cd49d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:41.924086 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9ce837-7f14-4f9c-a378-d2ca57cd49d5 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:41.936401 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 486/988] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 15:13:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:41.942420 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd7c7dc-7c3a-4537-9986-a89e1b2f5b97 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:41.947608 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:42.044957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd7c7dc-7c3a-4537-9986-a89e1b2f5b97 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:42.081299 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4050f9-d044-422f-8f59-9d557e04a687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:42.081921 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4050f9-d044-422f-8f59-9d557e04a687 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:42.082217 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4050f9-d044-422f-8f59-9d557e04a687 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:42.082489 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4050f9-d044-422f-8f59-9d557e04a687 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:42.083287 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4050f9-d044-422f-8f59-9d557e04a687 None admin] RBAC: Adding query filter params (role.id=e7638910be904dc388b948d044a90758, scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83, user.id=1654fb6a81e14d6b99ab495bb6d029d7) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:42.083603 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4050f9-d044-422f-8f59-9d557e04a687 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:42.085059 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4050f9-d044-422f-8f59-9d557e04a687 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:42.096267 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 503/989] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:13:41 2022] GET /identity/v3/role_assignments?user.id=1654fb6a81e14d6b99ab495bb6d029d7&scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83&role.id=e7638910be904dc388b948d044a90758 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:42.172976 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a09e4a-d554-4a07-9c22-815ddbdb90b5 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:42.173531 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a09e4a-d554-4a07-9c22-815ddbdb90b5 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:42.173531 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a09e4a-d554-4a07-9c22-815ddbdb90b5 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:42.175691 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 487/990] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:42.181031 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1574b75c-cb2a-4fcd-83d1-e555230565af None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:42.181330 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1574b75c-cb2a-4fcd-83d1-e555230565af None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:42.181609 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1574b75c-cb2a-4fcd-83d1-e555230565af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:42.201870 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-1574b75c-cb2a-4fcd-83d1-e555230565af None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:42.203183 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-1574b75c-cb2a-4fcd-83d1-e555230565af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:42.280976 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:42.285927 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 504/991] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:42 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:13:42.308941 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a09e4a-d554-4a07-9c22-815ddbdb90b5 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:42.419460 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a09e4a-d554-4a07-9c22-815ddbdb90b5 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:42.453238 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568f4782-4791-486f-98e3-47aed86c2b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:42.453881 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f4782-4791-486f-98e3-47aed86c2b58 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:42.454168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f4782-4791-486f-98e3-47aed86c2b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:42.454799 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f4782-4791-486f-98e3-47aed86c2b58 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:42.455438 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568f4782-4791-486f-98e3-47aed86c2b58 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:42.456845 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568f4782-4791-486f-98e3-47aed86c2b58 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:42.465818 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-568f4782-4791-486f-98e3-47aed86c2b58 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 15:13:42.465818 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:42.465818 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 15:13:42.465818 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 15:13:42.465818 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 15:13:42.465818 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 15:13:42.465818 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 15:13:42.465818 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:42.465818 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 15:13:42.465818 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:42.465818 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 15:13:42.465818 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:42.465818 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:42.465818 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:42.465818 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:42.465818 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:42.465818 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:42.466831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:42.466831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:42.466831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:42.466831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:42.466831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:42.466831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:42.466831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 15:13:42.466831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 15:13:42.466831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 15:13:42.466831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 15:13:42.466831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:42.466831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:42.466831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:42.466831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:42.466831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:42.466831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:42.467671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:42.467671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:42.467671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:42.467671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:42.467671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:42.467671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:42.467671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:42.467671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:42.467671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:42.467671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:42.467671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 15:13:42.467671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:42.467671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 15:13:42.467671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:42.467671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 488/992] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 5 15:13:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:42.474352 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1574b75c-cb2a-4fcd-83d1-e555230565af None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:42.576996 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1574b75c-cb2a-4fcd-83d1-e555230565af None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:42.617481 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042a0257-dc19-45a9-a2ba-179fb9c03112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:42.618159 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a0257-dc19-45a9-a2ba-179fb9c03112 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:42.618393 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a0257-dc19-45a9-a2ba-179fb9c03112 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:42.618655 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a0257-dc19-45a9-a2ba-179fb9c03112 None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:42.619341 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042a0257-dc19-45a9-a2ba-179fb9c03112 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:42.619648 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042a0257-dc19-45a9-a2ba-179fb9c03112 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:42.622112 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042a0257-dc19-45a9-a2ba-179fb9c03112 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:42.632758 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 505/993] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Aug 5 15:13:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:42.633853 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568f4782-4791-486f-98e3-47aed86c2b58 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:42.691306 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:42.736543 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568f4782-4791-486f-98e3-47aed86c2b58 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:42.770859 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b9097e-8ae5-4ec4-bf40-1a18e118e688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:42.771472 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9097e-8ae5-4ec4-bf40-1a18e118e688 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:42.771755 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9097e-8ae5-4ec4-bf40-1a18e118e688 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:42.772027 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9097e-8ae5-4ec4-bf40-1a18e118e688 None admin] PATH_INFO: `/v3/services` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:42.772739 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b9097e-8ae5-4ec4-bf40-1a18e118e688 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:42.773047 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b9097e-8ae5-4ec4-bf40-1a18e118e688 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:42.774399 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b9097e-8ae5-4ec4-bf40-1a18e118e688 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:42.784089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 489/994] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Aug 5 15:13:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:42.785396 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042a0257-dc19-45a9-a2ba-179fb9c03112 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:42.847246 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:42.886809 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042a0257-dc19-45a9-a2ba-179fb9c03112 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:42.925279 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a5b1ab-993f-4dc3-92f5-fd9979f8c4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:42.926805 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a5b1ab-993f-4dc3-92f5-fd9979f8c4d5 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:42.926805 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a5b1ab-993f-4dc3-92f5-fd9979f8c4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:42.926983 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a5b1ab-993f-4dc3-92f5-fd9979f8c4d5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:42.927620 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a5b1ab-993f-4dc3-92f5-fd9979f8c4d5 None admin] RBAC: Adding query filter params (interface=admin, service_id=3ecb213366f04902b3464f056d22ee52, region_id=RegionOne) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:42.927935 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a5b1ab-993f-4dc3-92f5-fd9979f8c4d5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:42.929893 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a5b1ab-993f-4dc3-92f5-fd9979f8c4d5 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:42.940807 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 506/995] 10.222.0.40 () {56 vars in 1249 bytes} [Fri Aug 5 15:13:42 2022] GET /identity/v3/endpoints?service_id=3ecb213366f04902b3464f056d22ee52&interface=admin®ion_id=RegionOne => generated 193 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:42.942315 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b9097e-8ae5-4ec4-bf40-1a18e118e688 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:43.041227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b9097e-8ae5-4ec4-bf40-1a18e118e688 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:43.058357 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:43.075340 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b69d51-0858-4174-82a9-3088d0f1985d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:43.075955 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b69d51-0858-4174-82a9-3088d0f1985d None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:43.076275 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b69d51-0858-4174-82a9-3088d0f1985d None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:43.076552 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b69d51-0858-4174-82a9-3088d0f1985d None admin] PATH_INFO: `/v3/services` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:43.077228 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b69d51-0858-4174-82a9-3088d0f1985d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:43.078583 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b69d51-0858-4174-82a9-3088d0f1985d None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:43.091084 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 490/996] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Aug 5 15:13:42 2022] GET /identity/v3/services => generated 1727 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 15:13:43.105836 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf4cf9-c4a7-4e8b-9ebe-862eb91fe178 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:43.106015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf4cf9-c4a7-4e8b-9ebe-862eb91fe178 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:43.106341 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf4cf9-c4a7-4e8b-9ebe-862eb91fe178 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:43.107770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 507/997] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:43.114122 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f381cd-0e05-4c9d-bdf2-c553e3e80da2 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:43.114455 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f381cd-0e05-4c9d-bdf2-c553e3e80da2 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:43.114670 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f381cd-0e05-4c9d-bdf2-c553e3e80da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:43.134248 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-77f381cd-0e05-4c9d-bdf2-c553e3e80da2 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:43.135554 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-77f381cd-0e05-4c9d-bdf2-c553e3e80da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:43.218591 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 491/998] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:43 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 15:13:43.241680 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bf4cf9-c4a7-4e8b-9ebe-862eb91fe178 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:43.355621 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bf4cf9-c4a7-4e8b-9ebe-862eb91fe178 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:43.382616 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:43.391464 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91df344-2985-4f8a-b49c-f748f7250c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:43.392221 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91df344-2985-4f8a-b49c-f748f7250c1d None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:43.392509 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91df344-2985-4f8a-b49c-f748f7250c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:43.392787 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91df344-2985-4f8a-b49c-f748f7250c1d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:43.393415 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91df344-2985-4f8a-b49c-f748f7250c1d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:43.394770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91df344-2985-4f8a-b49c-f748f7250c1d None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:43.403605 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-d91df344-2985-4f8a-b49c-f748f7250c1d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 15:13:43.403605 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:43.403605 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 15:13:43.403605 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 15:13:43.403605 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 15:13:43.403605 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 15:13:43.403605 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 15:13:43.403605 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:43.403605 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 15:13:43.403605 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:43.403605 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 15:13:43.403605 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:43.403605 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:43.403605 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:43.403605 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:43.403605 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:43.403605 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:43.407304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:43.407304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:43.407304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:43.407304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:43.407304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:43.407304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:43.407304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 15:13:43.407304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 15:13:43.407304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 15:13:43.407304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 15:13:43.407304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:43.407304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:43.407304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:43.407304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:43.407304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:43.407304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:43.408593 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:43.408593 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:43.408593 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:43.408593 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:43.408593 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:43.408593 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:43.408593 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:43.408593 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:43.408593 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:43.408593 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:43.408593 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 15:13:43.408593 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:43.408593 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 15:13:43.408593 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:43.408593 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 508/999] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:13:43 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:43.411183 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f381cd-0e05-4c9d-bdf2-c553e3e80da2 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:43.511823 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f381cd-0e05-4c9d-bdf2-c553e3e80da2 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:43.525970 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:43.550678 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fcd1b5-5f67-470f-8ebd-20ea97b4b276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:43.551308 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fcd1b5-5f67-470f-8ebd-20ea97b4b276 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:43.551592 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fcd1b5-5f67-470f-8ebd-20ea97b4b276 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:43.551872 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fcd1b5-5f67-470f-8ebd-20ea97b4b276 None admin] PATH_INFO: `/v3/services` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:43.552583 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fcd1b5-5f67-470f-8ebd-20ea97b4b276 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:43.552894 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fcd1b5-5f67-470f-8ebd-20ea97b4b276 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:43.554284 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fcd1b5-5f67-470f-8ebd-20ea97b4b276 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:43.563788 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 492/1000] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 5 15:13:43 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:43.565334 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91df344-2985-4f8a-b49c-f748f7250c1d None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:43.667169 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91df344-2985-4f8a-b49c-f748f7250c1d None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:43.700718 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97b668c-33a1-403c-abb1-eb008641d930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:43.701339 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b668c-33a1-403c-abb1-eb008641d930 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:43.701626 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b668c-33a1-403c-abb1-eb008641d930 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:43.701902 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b668c-33a1-403c-abb1-eb008641d930 None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:43.702612 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97b668c-33a1-403c-abb1-eb008641d930 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:43.702919 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97b668c-33a1-403c-abb1-eb008641d930 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:43.704313 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97b668c-33a1-403c-abb1-eb008641d930 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:43.713904 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 509/1001] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 5 15:13:43 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:43.715462 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fcd1b5-5f67-470f-8ebd-20ea97b4b276 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:43.819288 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fcd1b5-5f67-470f-8ebd-20ea97b4b276 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:43.854244 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b0b088-77cf-4574-b8dc-492c002793f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:43.854864 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b0b088-77cf-4574-b8dc-492c002793f4 None admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:43.855150 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b0b088-77cf-4574-b8dc-492c002793f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:43.855429 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b0b088-77cf-4574-b8dc-492c002793f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:43.856186 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b0b088-77cf-4574-b8dc-492c002793f4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:43.857555 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b0b088-77cf-4574-b8dc-492c002793f4 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:43.950919 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 493/1002] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:13:43 2022] POST /identity/v3/endpoints => generated 334 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:13:44.153666 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f853fc7-c38d-40b5-820b-803850a5bc68 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:44.153966 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f853fc7-c38d-40b5-820b-803850a5bc68 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:44.154249 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f853fc7-c38d-40b5-820b-803850a5bc68 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:44.156623 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 510/1003] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:44.162059 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a072b-ab4e-410e-bbcf-a69a855f2ba6 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:44.162357 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a072b-ab4e-410e-bbcf-a69a855f2ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:44.162640 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a072b-ab4e-410e-bbcf-a69a855f2ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:44.184171 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-d75a072b-ab4e-410e-bbcf-a69a855f2ba6 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:44.185388 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-d75a072b-ab4e-410e-bbcf-a69a855f2ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:44.217562 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:44.267396 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 494/1004] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:44 2022] POST /identity/v3/auth/tokens => generated 1726 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:13:44.289134 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f853fc7-c38d-40b5-820b-803850a5bc68 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:44.347738 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:44.394268 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f853fc7-c38d-40b5-820b-803850a5bc68 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:44.427345 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b104ec9-0533-4810-9f42-46e07a89d63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:44.427985 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b104ec9-0533-4810-9f42-46e07a89d63e None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:44.428299 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b104ec9-0533-4810-9f42-46e07a89d63e None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:44.428587 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b104ec9-0533-4810-9f42-46e07a89d63e None admin] PATH_INFO: `/v3/services/image` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:44.429216 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b104ec9-0533-4810-9f42-46e07a89d63e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:44.430579 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b104ec9-0533-4810-9f42-46e07a89d63e None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:44.439932 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-7b104ec9-0533-4810-9f42-46e07a89d63e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 15:13:44.439932 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:44.439932 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 15:13:44.439932 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 15:13:44.439932 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 15:13:44.439932 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 15:13:44.439932 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 15:13:44.439932 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:44.439932 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 15:13:44.439932 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:44.439932 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 15:13:44.439932 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:44.439932 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:44.439932 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:44.439932 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:44.439932 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:44.439932 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:44.440923 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:44.440923 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:44.440923 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:44.440923 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:44.440923 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:44.440923 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:44.440923 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 15:13:44.440923 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 15:13:44.440923 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 15:13:44.440923 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 15:13:44.440923 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:44.440923 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:44.440923 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:44.440923 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:44.440923 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:44.440923 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:44.447044 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:44.447044 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:44.447044 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:44.447044 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:44.447044 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:44.447044 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:44.447044 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:44.447044 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:44.447044 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:44.447044 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:44.447044 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 15:13:44.447044 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:44.447044 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 15:13:44.447044 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:44.447044 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 511/1005] 10.222.0.40 () {56 vars in 1105 bytes} [Fri Aug 5 15:13:44 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:44.449496 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75a072b-ab4e-410e-bbcf-a69a855f2ba6 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:44.562903 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75a072b-ab4e-410e-bbcf-a69a855f2ba6 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:44.597900 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5837ae8e-ceb8-41d5-a70a-0c2ec08dbd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:44.598526 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5837ae8e-ceb8-41d5-a70a-0c2ec08dbd92 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:44.598815 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5837ae8e-ceb8-41d5-a70a-0c2ec08dbd92 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:44.599095 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5837ae8e-ceb8-41d5-a70a-0c2ec08dbd92 None admin] PATH_INFO: `/v3/services` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:44.599777 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5837ae8e-ceb8-41d5-a70a-0c2ec08dbd92 None admin] RBAC: Adding query filter params (name=image) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:44.600110 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5837ae8e-ceb8-41d5-a70a-0c2ec08dbd92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:44.601474 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5837ae8e-ceb8-41d5-a70a-0c2ec08dbd92 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:44.606887 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 495/1006] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:44 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:44.612889 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b104ec9-0533-4810-9f42-46e07a89d63e None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:44.714465 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b104ec9-0533-4810-9f42-46e07a89d63e None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:44.731545 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:44.749840 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0074ea86-1fa9-4bc3-899d-910480e0447a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:44.750467 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074ea86-1fa9-4bc3-899d-910480e0447a None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:44.750755 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074ea86-1fa9-4bc3-899d-910480e0447a None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:44.751035 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074ea86-1fa9-4bc3-899d-910480e0447a None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:44.751717 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0074ea86-1fa9-4bc3-899d-910480e0447a None admin] RBAC: Adding query filter params (type=image) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:44.752036 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0074ea86-1fa9-4bc3-899d-910480e0447a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:44.753468 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0074ea86-1fa9-4bc3-899d-910480e0447a None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:44.758447 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 512/1007] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:44 2022] GET /identity/v3/services?type=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:45.126234 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252f887-5c52-4034-b2b6-dc33d57c7584 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:45.126581 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252f887-5c52-4034-b2b6-dc33d57c7584 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:45.126803 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252f887-5c52-4034-b2b6-dc33d57c7584 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:45.128324 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 496/1008] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:45.133280 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0cc36d-11f0-44be-b869-2c90248708f4 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:45.133580 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0cc36d-11f0-44be-b869-2c90248708f4 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:45.133863 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0cc36d-11f0-44be-b869-2c90248708f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:45.152578 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-cb0cc36d-11f0-44be-b869-2c90248708f4 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:45.153772 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-cb0cc36d-11f0-44be-b869-2c90248708f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:45.229903 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 513/1009] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:45 2022] POST /identity/v3/auth/tokens => generated 1726 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:13:45.253924 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0252f887-5c52-4034-b2b6-dc33d57c7584 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:45.279409 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:45.360690 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0252f887-5c52-4034-b2b6-dc33d57c7584 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:45.395665 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad768f-b0b8-40a9-ad6a-2aea36baa06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:45.396270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad768f-b0b8-40a9-ad6a-2aea36baa06f None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:45.396560 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad768f-b0b8-40a9-ad6a-2aea36baa06f None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:45.396838 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad768f-b0b8-40a9-ad6a-2aea36baa06f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:45.397464 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cad768f-b0b8-40a9-ad6a-2aea36baa06f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:45.398827 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cad768f-b0b8-40a9-ad6a-2aea36baa06f None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:45.408421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-1cad768f-b0b8-40a9-ad6a-2aea36baa06f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 15:13:45.408421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:45.408421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 15:13:45.408421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 15:13:45.408421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 15:13:45.408421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 15:13:45.408421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 15:13:45.408421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:45.408421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 15:13:45.408421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:45.408421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 15:13:45.408421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:45.408421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:45.408421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:45.408421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:45.408421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:45.408421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:45.409380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:45.409380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:45.409380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:45.409380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:45.409380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:45.409380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:45.409380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 15:13:45.409380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 15:13:45.409380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 15:13:45.409380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 15:13:45.409380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:45.409380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:45.409380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:45.409380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:45.409380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:45.409380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:45.410243 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:45.410243 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:45.410243 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:45.410243 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:45.410243 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:45.410243 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:45.410243 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:45.410243 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:45.410243 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:45.410243 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:45.410243 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 15:13:45.410243 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:45.410243 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 15:13:45.410243 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:45.411639 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 497/1010] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 5 15:13:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:45.415947 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0cc36d-11f0-44be-b869-2c90248708f4 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:45.517186 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0cc36d-11f0-44be-b869-2c90248708f4 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:45.530959 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:45.552555 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03953286-13ef-426a-8bbd-ed2f58b28de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:45.553173 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03953286-13ef-426a-8bbd-ed2f58b28de7 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:45.553454 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03953286-13ef-426a-8bbd-ed2f58b28de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:45.553732 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03953286-13ef-426a-8bbd-ed2f58b28de7 None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:45.554483 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03953286-13ef-426a-8bbd-ed2f58b28de7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:45.555204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03953286-13ef-426a-8bbd-ed2f58b28de7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:45.557316 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03953286-13ef-426a-8bbd-ed2f58b28de7 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:45.566762 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 514/1011] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Aug 5 15:13:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:45.568340 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad768f-b0b8-40a9-ad6a-2aea36baa06f None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:45.658018 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:45.672622 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad768f-b0b8-40a9-ad6a-2aea36baa06f None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:45.708477 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af47dd72-7315-4853-9e99-3b2069c7c89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:45.709101 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47dd72-7315-4853-9e99-3b2069c7c89d None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:45.709390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47dd72-7315-4853-9e99-3b2069c7c89d None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:45.709669 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47dd72-7315-4853-9e99-3b2069c7c89d None admin] PATH_INFO: `/v3/services` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:45.710371 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af47dd72-7315-4853-9e99-3b2069c7c89d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:45.710656 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af47dd72-7315-4853-9e99-3b2069c7c89d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:45.712018 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af47dd72-7315-4853-9e99-3b2069c7c89d None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:45.717843 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 498/1012] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Aug 5 15:13:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:45.723525 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03953286-13ef-426a-8bbd-ed2f58b28de7 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:45.830231 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03953286-13ef-426a-8bbd-ed2f58b28de7 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:45.866791 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63df28dd-8e9a-424f-93ed-81440899e455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:45.867420 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df28dd-8e9a-424f-93ed-81440899e455 None admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:45.867710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df28dd-8e9a-424f-93ed-81440899e455 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:45.867990 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df28dd-8e9a-424f-93ed-81440899e455 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:45.868795 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63df28dd-8e9a-424f-93ed-81440899e455 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:45.870171 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63df28dd-8e9a-424f-93ed-81440899e455 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:45.922129 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 515/1013] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:13:45 2022] POST /identity/v3/endpoints => generated 325 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:13:46.090538 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617ef78-1c30-40f8-bcb6-088669449ce9 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:46.090894 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617ef78-1c30-40f8-bcb6-088669449ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:46.091176 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617ef78-1c30-40f8-bcb6-088669449ce9 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:46.093436 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 499/1014] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:46.098399 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d5d061-c3d1-4b96-b192-69e026860534 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:46.098698 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d5d061-c3d1-4b96-b192-69e026860534 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:46.098986 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d5d061-c3d1-4b96-b192-69e026860534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:46.117764 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-f4d5d061-c3d1-4b96-b192-69e026860534 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:46.118954 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-f4d5d061-c3d1-4b96-b192-69e026860534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:46.150049 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:46.205299 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 516/1015] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:46 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 15:13:46.232870 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0617ef78-1c30-40f8-bcb6-088669449ce9 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:46.339665 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0617ef78-1c30-40f8-bcb6-088669449ce9 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:46.374510 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821e5e03-034e-4311-840c-bec9bc447088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:46.375161 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e5e03-034e-4311-840c-bec9bc447088 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:46.375445 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e5e03-034e-4311-840c-bec9bc447088 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:46.375726 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e5e03-034e-4311-840c-bec9bc447088 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:46.386732 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821e5e03-034e-4311-840c-bec9bc447088 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:13:46.392368 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-821e5e03-034e-4311-840c-bec9bc447088 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 15:13:46.392368 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:46.392368 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:46.392368 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:46.392368 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:46.392368 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:46.392368 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:46.392368 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:46.392368 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:46.392368 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:46.392368 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:46.392368 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:46.392368 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:13:46.392368 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:13:46.392368 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:13:46.392368 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:13:46.393321 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:13:46.393321 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:13:46.393321 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:13:46.393321 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:13:46.393321 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:46.393321 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:46.393321 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:46.393321 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:46.393321 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:46.393321 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:46.393321 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:46.393321 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:46.393321 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:46.393321 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:46.393321 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:46.393321 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:46.394177 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:46.394177 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:46.394177 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:46.394177 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:46.394177 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:13:46.394177 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:13:46.394177 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:13:46.394177 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:13:46.394177 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:13:46.394177 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:13:46.394177 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 15:13:46.394177 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:46.396409 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 500/1016] 10.222.0.40 () {56 vars in 1120 bytes} [Fri Aug 5 15:13:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:46.399868 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d5d061-c3d1-4b96-b192-69e026860534 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:46.502226 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d5d061-c3d1-4b96-b192-69e026860534 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:46.536639 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4924277c-9763-4f36-8021-42031b50ec3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:46.537247 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4924277c-9763-4f36-8021-42031b50ec3e None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:46.537537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4924277c-9763-4f36-8021-42031b50ec3e None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:46.537817 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4924277c-9763-4f36-8021-42031b50ec3e None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:46.538539 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4924277c-9763-4f36-8021-42031b50ec3e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:46.538851 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4924277c-9763-4f36-8021-42031b50ec3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:46.540252 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4924277c-9763-4f36-8021-42031b50ec3e None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:46.553811 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 517/1017] 10.222.0.40 () {56 vars in 1115 bytes} [Fri Aug 5 15:13:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:46.554984 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821e5e03-034e-4311-840c-bec9bc447088 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:46.568342 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:46.657268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821e5e03-034e-4311-840c-bec9bc447088 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:46.690892 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36c851-03dc-479f-996a-8607ebbee357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:46.691531 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36c851-03dc-479f-996a-8607ebbee357 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:46.691818 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36c851-03dc-479f-996a-8607ebbee357 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:46.692123 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36c851-03dc-479f-996a-8607ebbee357 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:46.703282 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad36c851-03dc-479f-996a-8607ebbee357 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:46.704770 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad36c851-03dc-479f-996a-8607ebbee357 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:46.718802 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-ad36c851-03dc-479f-996a-8607ebbee357 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:46.718802 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:46.718802 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:46.718802 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:46.718802 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:46.718802 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:46.718802 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:46.718802 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:46.718802 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:46.718802 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:46.718802 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:46.718802 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:46.718802 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:46.718802 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:46.718802 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:46.719692 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:46.719692 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:46.719692 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:46.719692 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:46.719692 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:46.719692 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:46.719692 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:46.719692 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:46.719692 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:46.719692 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:46.719692 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:46.719692 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:46.719692 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:46.719692 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:46.719692 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:46.719692 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:46.719692 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:46.730825 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:46.730825 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:46.730825 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:46.730825 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:46.730825 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 501/1018] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:13:46.731476 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4924277c-9763-4f36-8021-42031b50ec3e None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:46.756643 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:46.760715 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf34d95-ba0d-4a31-8a14-c462c6638ece None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:46.761000 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf34d95-ba0d-4a31-8a14-c462c6638ece None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:46.761280 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf34d95-ba0d-4a31-8a14-c462c6638ece None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:46.762693 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 502/1019] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 15:13:46.767833 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d9108-f595-456c-97f4-9d92ed653dc6 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:46.768161 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d9108-f595-456c-97f4-9d92ed653dc6 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:46.768451 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d9108-f595-456c-97f4-9d92ed653dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:46.787511 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-056d9108-f595-456c-97f4-9d92ed653dc6 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:46.788733 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-056d9108-f595-456c-97f4-9d92ed653dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:46.835550 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4924277c-9763-4f36-8021-42031b50ec3e None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:46.845825 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:46.857568 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 503/1020] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:46 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 15:13:46.874403 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e95d5-1a75-4209-8d45-b418ac59b2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:46.875254 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e95d5-1a75-4209-8d45-b418ac59b2dc None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:46.875677 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e95d5-1a75-4209-8d45-b418ac59b2dc None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:46.876148 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e95d5-1a75-4209-8d45-b418ac59b2dc None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:46.877103 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296e95d5-1a75-4209-8d45-b418ac59b2dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:46.877551 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296e95d5-1a75-4209-8d45-b418ac59b2dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:46.879127 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296e95d5-1a75-4209-8d45-b418ac59b2dc None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:46.881307 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d9108-f595-456c-97f4-9d92ed653dc6 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:46.899183 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 518/1021] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:46.900434 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e95d5-1a75-4209-8d45-b418ac59b2dc None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:47.007710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:47.008202 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d9108-f595-456c-97f4-9d92ed653dc6 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:47.027366 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e95d5-1a75-4209-8d45-b418ac59b2dc None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:47.044784 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacaf4fd-aff9-49c5-81ab-3a65039f258f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:47.046000 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacaf4fd-aff9-49c5-81ab-3a65039f258f None admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:47.046290 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacaf4fd-aff9-49c5-81ab-3a65039f258f None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:47.046569 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacaf4fd-aff9-49c5-81ab-3a65039f258f None admin] PATH_INFO: `/v3/services` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:47.047267 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacaf4fd-aff9-49c5-81ab-3a65039f258f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:47.048683 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacaf4fd-aff9-49c5-81ab-3a65039f258f None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:47.070060 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f2ff6b-d47f-461e-afb0-48be92c7fdd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:47.070692 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2ff6b-d47f-461e-afb0-48be92c7fdd8 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:47.070979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2ff6b-d47f-461e-afb0-48be92c7fdd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:47.071258 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2ff6b-d47f-461e-afb0-48be92c7fdd8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:47.078494 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f2ff6b-d47f-461e-afb0-48be92c7fdd8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:47.079952 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f2ff6b-d47f-461e-afb0-48be92c7fdd8 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:47.091488 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-45f2ff6b-d47f-461e-afb0-48be92c7fdd8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 15:13:47.091488 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:47.091488 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:47.091488 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:47.091488 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:47.091488 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:47.091488 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:47.091488 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:47.091488 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:47.091488 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:47.091488 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:47.091488 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:47.091488 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 15:13:47.091488 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 15:13:47.091488 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 15:13:47.091488 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 15:13:47.097384 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:47.097384 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:47.097384 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 15:13:47.097384 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:47.097384 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 15:13:47.097384 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:47.097384 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:47.097384 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:47.097384 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:47.097384 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:47.097384 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:47.097384 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:47.097384 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:47.097384 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:47.097384 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:47.097384 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:47.097384 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:47.097384 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:47.098576 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:47.098576 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:47.098576 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 15:13:47.098576 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 15:13:47.098576 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 15:13:47.098576 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 15:13:47.098576 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 15:13:47.098576 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 15:13:47.098576 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 15:13:47.098576 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:47.098576 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 519/1022] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Aug 5 15:13:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:47.098576 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f2ff6b-d47f-461e-afb0-48be92c7fdd8 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:47.110827 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 504/1023] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 5 15:13:46 2022] POST /identity/v3/services => generated 242 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:13:47.223221 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f2ff6b-d47f-461e-afb0-48be92c7fdd8 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:47.277220 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83741edc-b9af-4873-854e-9833590dda1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:47.278168 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83741edc-b9af-4873-854e-9833590dda1d None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:47.278634 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83741edc-b9af-4873-854e-9833590dda1d None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:47.279096 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83741edc-b9af-4873-854e-9833590dda1d None admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:47.280443 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83741edc-b9af-4873-854e-9833590dda1d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:47.280763 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83741edc-b9af-4873-854e-9833590dda1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:47.282209 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83741edc-b9af-4873-854e-9833590dda1d None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:47.301245 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:47.313221 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 520/1024] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Aug 5 15:13:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:47.319031 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacaf4fd-aff9-49c5-81ab-3a65039f258f None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:47.374762 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:47.423364 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacaf4fd-aff9-49c5-81ab-3a65039f258f None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:47.457298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f7797-f237-4998-a5a4-37f06c3ae708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:47.457932 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f7797-f237-4998-a5a4-37f06c3ae708 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:47.458220 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f7797-f237-4998-a5a4-37f06c3ae708 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:47.458500 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f7797-f237-4998-a5a4-37f06c3ae708 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:47.526794 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f7797-f237-4998-a5a4-37f06c3ae708 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:47.528341 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f7797-f237-4998-a5a4-37f06c3ae708 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:47.584252 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:47.644542 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 505/1025] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:13:47 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 15:13:47.645681 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83741edc-b9af-4873-854e-9833590dda1d None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:47.746886 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83741edc-b9af-4873-854e-9833590dda1d None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:47.781997 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8635c773-d70e-48e6-83d4-3a2bd6ccf11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:47.782568 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635c773-d70e-48e6-83d4-3a2bd6ccf11a None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:47.782854 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635c773-d70e-48e6-83d4-3a2bd6ccf11a None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:47.783133 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635c773-d70e-48e6-83d4-3a2bd6ccf11a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:47.793504 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8635c773-d70e-48e6-83d4-3a2bd6ccf11a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:47.794956 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8635c773-d70e-48e6-83d4-3a2bd6ccf11a None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:47.809688 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-8635c773-d70e-48e6-83d4-3a2bd6ccf11a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:47.809688 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:47.809688 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:47.809688 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:47.809688 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:47.809688 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:47.809688 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:47.809688 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:47.809688 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:47.809688 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:47.809688 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:47.809688 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:47.809688 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:47.809688 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:47.809688 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:47.810708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:47.810708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:47.810708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:47.810708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:47.810708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:47.810708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:47.810708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:47.810708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:47.810708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:47.810708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:47.810708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:47.810708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:47.810708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:47.810708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:47.810708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:47.810708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:47.810708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:47.811641 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:47.811641 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:47.811641 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:47.811641 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:47.811641 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 521/1026] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:13:47.817631 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f7797-f237-4998-a5a4-37f06c3ae708 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:47.921034 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f7797-f237-4998-a5a4-37f06c3ae708 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:47.949711 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3600a7d7-9622-43bf-be3a-ebcae257ddc3 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:47.950054 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3600a7d7-9622-43bf-be3a-ebcae257ddc3 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:47.950268 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3600a7d7-9622-43bf-be3a-ebcae257ddc3 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:47.952393 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 522/1027] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 15:13:47.956776 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037c7e3f-e1d5-42fe-8685-211e7f8433d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:47.957114 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23fe3f-31bd-41e5-a0af-2799f9f08199 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:47.957394 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c7e3f-e1d5-42fe-8685-211e7f8433d7 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:47.957495 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23fe3f-31bd-41e5-a0af-2799f9f08199 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:47.957662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23fe3f-31bd-41e5-a0af-2799f9f08199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:47.957772 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c7e3f-e1d5-42fe-8685-211e7f8433d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:47.957947 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c7e3f-e1d5-42fe-8685-211e7f8433d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:47.958635 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037c7e3f-e1d5-42fe-8685-211e7f8433d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:47.958946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037c7e3f-e1d5-42fe-8685-211e7f8433d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:47.960322 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037c7e3f-e1d5-42fe-8685-211e7f8433d7 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:47.976163 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 506/1028] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:47.977208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037c7e3f-e1d5-42fe-8685-211e7f8433d7 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:47.980559 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-4d23fe3f-31bd-41e5-a0af-2799f9f08199 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:47.981776 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-4d23fe3f-31bd-41e5-a0af-2799f9f08199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:47.992027 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:48.029312 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:48.050688 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 523/1029] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:47 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 15:13:48.073725 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d23fe3f-31bd-41e5-a0af-2799f9f08199 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:48.083466 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037c7e3f-e1d5-42fe-8685-211e7f8433d7 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:48.118613 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd98d2-f3d9-48d7-88be-1a8858ebba3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:48.119259 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd98d2-f3d9-48d7-88be-1a8858ebba3f None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:48.119587 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd98d2-f3d9-48d7-88be-1a8858ebba3f None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:48.119827 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd98d2-f3d9-48d7-88be-1a8858ebba3f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:48.127543 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bd98d2-f3d9-48d7-88be-1a8858ebba3f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:48.129014 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bd98d2-f3d9-48d7-88be-1a8858ebba3f None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:48.140830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-26bd98d2-f3d9-48d7-88be-1a8858ebba3f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:48.140830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:48.140830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:48.140830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:48.140830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:48.140830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:48.140830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:48.140830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:48.140830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:48.140830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:48.140830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:48.140830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:48.140830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 15:13:48.140830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 15:13:48.140830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 15:13:48.141718 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 15:13:48.141718 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:48.141718 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:48.141718 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:48.141718 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:48.141718 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:48.141718 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:48.141718 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:48.141718 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:48.141718 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:48.141718 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:48.141718 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:48.141718 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:48.141718 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:48.141718 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:48.141718 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:48.141718 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:48.142613 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 15:13:48.142613 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 15:13:48.142613 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 15:13:48.142613 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 15:13:48.142613 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 15:13:48.142613 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 15:13:48.142613 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:48.142613 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:48.142613 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 507/1030] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:13:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:48.147696 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd98d2-f3d9-48d7-88be-1a8858ebba3f None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:48.189254 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d23fe3f-31bd-41e5-a0af-2799f9f08199 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:48.230671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:48.233438 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840e21af-a99f-4468-bb15-c5cbe3517d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:48.234237 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e21af-a99f-4468-bb15-c5cbe3517d2f None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:48.234530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e21af-a99f-4468-bb15-c5cbe3517d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:48.234809 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e21af-a99f-4468-bb15-c5cbe3517d2f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:48.236124 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840e21af-a99f-4468-bb15-c5cbe3517d2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:48.237573 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840e21af-a99f-4468-bb15-c5cbe3517d2f None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:48.240111 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 524/1031] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:48.245388 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840e21af-a99f-4468-bb15-c5cbe3517d2f None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:48.265153 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd98d2-f3d9-48d7-88be-1a8858ebba3f None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:48.301263 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:48.301662 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1cc17d-1cad-403d-b34f-355db4ccb335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:48.302473 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1cc17d-1cad-403d-b34f-355db4ccb335 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:48.302928 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1cc17d-1cad-403d-b34f-355db4ccb335 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:48.303204 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1cc17d-1cad-403d-b34f-355db4ccb335 None admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:48.303949 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1cc17d-1cad-403d-b34f-355db4ccb335 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:48.304290 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1cc17d-1cad-403d-b34f-355db4ccb335 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:48.305705 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1cc17d-1cad-403d-b34f-355db4ccb335 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:48.323411 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 508/1032] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 15:13:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:48.324463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1cc17d-1cad-403d-b34f-355db4ccb335 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:48.363993 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840e21af-a99f-4468-bb15-c5cbe3517d2f None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:48.403751 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35169b12-628b-4cde-adf7-bd2ac6f2038c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:48.404407 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35169b12-628b-4cde-adf7-bd2ac6f2038c None admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:48.404730 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35169b12-628b-4cde-adf7-bd2ac6f2038c None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:48.405204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35169b12-628b-4cde-adf7-bd2ac6f2038c None admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:48.405692 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35169b12-628b-4cde-adf7-bd2ac6f2038c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:48.407132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35169b12-628b-4cde-adf7-bd2ac6f2038c None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:48.428883 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1cc17d-1cad-403d-b34f-355db4ccb335 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:48.458511 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:48.463755 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0ddf66-a55f-432a-819b-034043d4fe6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:48.464398 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ddf66-a55f-432a-819b-034043d4fe6e None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:48.464686 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ddf66-a55f-432a-819b-034043d4fe6e None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:48.464966 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ddf66-a55f-432a-819b-034043d4fe6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:48.465748 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0ddf66-a55f-432a-819b-034043d4fe6e None admin] RBAC: Adding query filter params (role.id=e7638910be904dc388b948d044a90758, scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83, user.id=bc0497dab1f44bb7b8196d39bf01d770) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:48.466065 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0ddf66-a55f-432a-819b-034043d4fe6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:48.467653 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0ddf66-a55f-432a-819b-034043d4fe6e None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:48.473006 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 509/1033] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:13:48 2022] GET /identity/v3/role_assignments?user.id=bc0497dab1f44bb7b8196d39bf01d770&scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83&role.id=e7638910be904dc388b948d044a90758 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:48.500579 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 525/1034] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 5 15:13:48 2022] POST /identity/v3/projects => generated 311 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:13:49.177267 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45936fd-e836-4512-975b-f461c02d5b6a None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:49.177567 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45936fd-e836-4512-975b-f461c02d5b6a None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:49.177849 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45936fd-e836-4512-975b-f461c02d5b6a None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:49.179384 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 510/1035] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:49.184325 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479ad53-abd6-49b0-815f-3bba26df8fc3 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:49.184748 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479ad53-abd6-49b0-815f-3bba26df8fc3 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:49.184914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479ad53-abd6-49b0-815f-3bba26df8fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:49.203556 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-e479ad53-abd6-49b0-815f-3bba26df8fc3 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:49.204909 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-e479ad53-abd6-49b0-815f-3bba26df8fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:49.276402 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 526/1036] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:49 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:13:49.299188 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45936fd-e836-4512-975b-f461c02d5b6a None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:49.398122 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45936fd-e836-4512-975b-f461c02d5b6a None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:49.430784 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d211904-6fb8-43ed-b8a0-e6be1fa33368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:49.431355 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d211904-6fb8-43ed-b8a0-e6be1fa33368 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:49.431620 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d211904-6fb8-43ed-b8a0-e6be1fa33368 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:49.431899 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d211904-6fb8-43ed-b8a0-e6be1fa33368 None admin] PATH_INFO: `/v3/services/image` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:49.432551 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d211904-6fb8-43ed-b8a0-e6be1fa33368 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:49.433901 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d211904-6fb8-43ed-b8a0-e6be1fa33368 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:49.443196 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-0d211904-6fb8-43ed-b8a0-e6be1fa33368 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 15:13:49.443196 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:49.443196 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 15:13:49.443196 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 15:13:49.443196 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 15:13:49.443196 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 15:13:49.443196 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 15:13:49.443196 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:49.443196 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 15:13:49.443196 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:49.443196 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 15:13:49.443196 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:49.443196 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:49.443196 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:49.443196 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:49.443196 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:49.443196 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:49.444230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:49.444230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:49.444230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:49.444230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:49.444230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:49.444230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:49.444230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 15:13:49.444230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 15:13:49.444230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 15:13:49.444230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 15:13:49.444230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:49.444230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:49.444230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:49.444230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:49.444230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:49.444230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:49.445087 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:49.445087 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:49.445087 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:49.445087 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:49.445087 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:49.445087 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:49.445087 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:49.445087 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:49.445087 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:49.445087 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:49.445087 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 15:13:49.445087 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:49.445087 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 15:13:49.445087 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:49.448384 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 511/1037] 10.222.0.40 () {56 vars in 1105 bytes} [Fri Aug 5 15:13:49 2022] GET /identity/v3/services/image => generated 86 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:49.450229 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e479ad53-abd6-49b0-815f-3bba26df8fc3 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:49.480375 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:49.551693 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e479ad53-abd6-49b0-815f-3bba26df8fc3 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:49.588595 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f69260-3a05-491a-813c-7463232e9cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:49.589240 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f69260-3a05-491a-813c-7463232e9cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:49.589528 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f69260-3a05-491a-813c-7463232e9cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:49.589804 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f69260-3a05-491a-813c-7463232e9cf5 None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:49.590501 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f69260-3a05-491a-813c-7463232e9cf5 None admin] RBAC: Adding query filter params (name=image) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:49.590813 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f69260-3a05-491a-813c-7463232e9cf5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:49.592246 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f69260-3a05-491a-813c-7463232e9cf5 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:49.602535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 527/1038] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:49 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:49.603944 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d211904-6fb8-43ed-b8a0-e6be1fa33368 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:49.667363 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:49.709365 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d211904-6fb8-43ed-b8a0-e6be1fa33368 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:49.742006 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955fb22a-df1d-4b15-a37e-1c5a3d339c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:49.742621 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955fb22a-df1d-4b15-a37e-1c5a3d339c4f None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:49.742908 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955fb22a-df1d-4b15-a37e-1c5a3d339c4f None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:49.743185 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955fb22a-df1d-4b15-a37e-1c5a3d339c4f None admin] PATH_INFO: `/v3/services` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:49.743856 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955fb22a-df1d-4b15-a37e-1c5a3d339c4f None admin] RBAC: Adding query filter params (type=image) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:49.744201 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955fb22a-df1d-4b15-a37e-1c5a3d339c4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:49.745581 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955fb22a-df1d-4b15-a37e-1c5a3d339c4f None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:49.756188 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 512/1039] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:49 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:49.757157 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f69260-3a05-491a-813c-7463232e9cf5 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:49.850174 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:49.857270 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f69260-3a05-491a-813c-7463232e9cf5 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:49.891573 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e806919e-cfb4-4fd0-a1ab-8444e13f1eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:49.892249 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e806919e-cfb4-4fd0-a1ab-8444e13f1eed None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:49.892536 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e806919e-cfb4-4fd0-a1ab-8444e13f1eed None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:49.892815 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e806919e-cfb4-4fd0-a1ab-8444e13f1eed None admin] PATH_INFO: `/v3/endpoints` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:49.893568 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e806919e-cfb4-4fd0-a1ab-8444e13f1eed None admin] RBAC: Adding query filter params (interface=public, service_id=62458cf9b33d4289861ce4b51b0b6261, region_id=RegionOne) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:49.893884 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e806919e-cfb4-4fd0-a1ab-8444e13f1eed None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:49.895254 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e806919e-cfb4-4fd0-a1ab-8444e13f1eed None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:49.905120 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 528/1040] 10.222.0.40 () {56 vars in 1251 bytes} [Fri Aug 5 15:13:49 2022] GET /identity/v3/endpoints?service_id=62458cf9b33d4289861ce4b51b0b6261&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:49.905946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955fb22a-df1d-4b15-a37e-1c5a3d339c4f None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:50.006560 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955fb22a-df1d-4b15-a37e-1c5a3d339c4f None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:50.029256 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:50.040234 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae8a635-c6ff-412a-a014-dece76165b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:50.040867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae8a635-c6ff-412a-a014-dece76165b88 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:50.041149 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae8a635-c6ff-412a-a014-dece76165b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:50.041424 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae8a635-c6ff-412a-a014-dece76165b88 None admin] PATH_INFO: `/v3/services` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:50.042078 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae8a635-c6ff-412a-a014-dece76165b88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:50.043435 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae8a635-c6ff-412a-a014-dece76165b88 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:50.056097 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 513/1041] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Aug 5 15:13:49 2022] GET /identity/v3/services => generated 1957 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 15:13:50.495222 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a97025-e4b7-4386-bc69-a077ac9c6770 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:50.495711 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a97025-e4b7-4386-bc69-a077ac9c6770 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:50.495711 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a97025-e4b7-4386-bc69-a077ac9c6770 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:50.497400 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 529/1042] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:50.502541 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f505231d-c402-45d1-b29d-253da77e02ce None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:50.502945 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f505231d-c402-45d1-b29d-253da77e02ce None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:50.503112 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f505231d-c402-45d1-b29d-253da77e02ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:50.521655 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-f505231d-c402-45d1-b29d-253da77e02ce None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:50.526662 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-f505231d-c402-45d1-b29d-253da77e02ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:50.565913 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397a129-424d-4d96-823f-1b5eeeec4e1b None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:50.566207 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397a129-424d-4d96-823f-1b5eeeec4e1b None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:50.566483 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397a129-424d-4d96-823f-1b5eeeec4e1b None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:50.568000 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 530/1043] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 15:13:50.573946 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb939b04-d57e-4364-ae10-0a70ff1df008 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:50.574238 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb939b04-d57e-4364-ae10-0a70ff1df008 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:50.574520 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb939b04-d57e-4364-ae10-0a70ff1df008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:50.600339 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-eb939b04-d57e-4364-ae10-0a70ff1df008 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:50.600521 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 514/1044] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:50 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:13:50.600625 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-eb939b04-d57e-4364-ae10-0a70ff1df008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:50.617871 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f505231d-c402-45d1-b29d-253da77e02ce None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:50.671456 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 531/1045] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:50 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 15:13:50.695208 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb939b04-d57e-4364-ae10-0a70ff1df008 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:50.724266 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f505231d-c402-45d1-b29d-253da77e02ce None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:50.742962 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:50.761227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c8e34-27d9-4281-8de0-214efc68038f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:50.761227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c8e34-27d9-4281-8de0-214efc68038f None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:50.761480 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c8e34-27d9-4281-8de0-214efc68038f None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:50.761688 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c8e34-27d9-4281-8de0-214efc68038f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:50.772924 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70c8e34-27d9-4281-8de0-214efc68038f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:13:50.778120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-e70c8e34-27d9-4281-8de0-214efc68038f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 15:13:50.778120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:50.778120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:50.778120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:50.778120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:50.778120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:50.778120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:50.778120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:50.778120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:50.778120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:50.778120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:50.778120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:50.778120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:13:50.778120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:13:50.778120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:13:50.778120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:13:50.779233 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:13:50.779233 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:13:50.779233 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:13:50.779233 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:13:50.779233 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:50.779233 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:50.779233 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:50.779233 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:50.779233 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:50.779233 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:50.779233 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:50.779233 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:50.779233 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:50.779233 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:50.779233 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:50.779233 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:50.780921 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:50.780921 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:50.780921 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:50.780921 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:50.780921 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:13:50.780921 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:13:50.780921 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:13:50.780921 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:13:50.780921 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:13:50.780921 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:13:50.780921 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 15:13:50.780921 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:50.784794 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 515/1046] 10.222.0.40 () {56 vars in 1120 bytes} [Fri Aug 5 15:13:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:13:50.785634 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c8e34-27d9-4281-8de0-214efc68038f None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:50.790358 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:50.802585 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb939b04-d57e-4364-ae10-0a70ff1df008 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:50.838101 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f78115-f3d2-4d3e-81bd-f6e1663f6e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:50.838746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f78115-f3d2-4d3e-81bd-f6e1663f6e6d None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:50.839029 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f78115-f3d2-4d3e-81bd-f6e1663f6e6d None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:50.839301 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f78115-f3d2-4d3e-81bd-f6e1663f6e6d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:50.840607 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f78115-f3d2-4d3e-81bd-f6e1663f6e6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:50.842622 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f78115-f3d2-4d3e-81bd-f6e1663f6e6d None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:50.848096 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 532/1047] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:50.849863 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f78115-f3d2-4d3e-81bd-f6e1663f6e6d None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:50.898887 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c8e34-27d9-4281-8de0-214efc68038f None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:50.937202 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e0a763-c8e2-47dd-88dd-afffab86f5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:50.937826 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0a763-c8e2-47dd-88dd-afffab86f5f4 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:50.938093 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0a763-c8e2-47dd-88dd-afffab86f5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:50.938484 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0a763-c8e2-47dd-88dd-afffab86f5f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:50.939192 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e0a763-c8e2-47dd-88dd-afffab86f5f4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:50.939501 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e0a763-c8e2-47dd-88dd-afffab86f5f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:50.940885 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e0a763-c8e2-47dd-88dd-afffab86f5f4 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:50.954718 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 516/1048] 10.222.0.40 () {56 vars in 1115 bytes} [Fri Aug 5 15:13:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:50.954885 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f78115-f3d2-4d3e-81bd-f6e1663f6e6d None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:50.955686 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e0a763-c8e2-47dd-88dd-afffab86f5f4 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:50.989591 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca8fe2a-4f21-4724-9708-c5f425d6d063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:50.990204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca8fe2a-4f21-4724-9708-c5f425d6d063 None admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:50.990486 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca8fe2a-4f21-4724-9708-c5f425d6d063 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:50.990764 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca8fe2a-4f21-4724-9708-c5f425d6d063 None admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:50.991447 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca8fe2a-4f21-4724-9708-c5f425d6d063 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:50.992897 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca8fe2a-4f21-4724-9708-c5f425d6d063 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:50.998792 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:51.056883 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e0a763-c8e2-47dd-88dd-afffab86f5f4 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:51.087396 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 533/1049] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 5 15:13:50 2022] POST /identity/v3/users => generated 285 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:13:51.100007 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59202110-5535-484c-8f31-c0640a7f68be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:51.100759 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59202110-5535-484c-8f31-c0640a7f68be None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:51.101042 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59202110-5535-484c-8f31-c0640a7f68be None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:51.101318 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59202110-5535-484c-8f31-c0640a7f68be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:51.115735 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59202110-5535-484c-8f31-c0640a7f68be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:51.115735 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59202110-5535-484c-8f31-c0640a7f68be None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:51.132479 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-59202110-5535-484c-8f31-c0640a7f68be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:51.132479 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:51.132479 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:51.132479 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:51.132479 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:51.132479 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:51.132479 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:51.132479 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:51.132479 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:51.132479 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:51.132479 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:51.132479 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:51.132479 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:51.132479 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:51.132479 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:51.133486 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:51.133486 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:51.133486 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:51.133486 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:51.133486 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:51.133486 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:51.133486 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:51.133486 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:51.133486 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:51.133486 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:51.133486 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:51.133486 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:51.133486 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:51.133486 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:51.133486 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:51.133486 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:51.133486 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:51.134401 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:51.134401 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:51.134401 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:51.134401 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:51.134401 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 517/1050] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:13:51.140359 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca8fe2a-4f21-4724-9708-c5f425d6d063 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:51.157430 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:51.259386 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca8fe2a-4f21-4724-9708-c5f425d6d063 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:51.294703 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acb776d-d370-41d0-a5cc-692a8469fac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:51.295847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb776d-d370-41d0-a5cc-692a8469fac9 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:51.296143 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb776d-d370-41d0-a5cc-692a8469fac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:51.296427 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb776d-d370-41d0-a5cc-692a8469fac9 None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:51.297127 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acb776d-d370-41d0-a5cc-692a8469fac9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:51.297441 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acb776d-d370-41d0-a5cc-692a8469fac9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:51.298813 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acb776d-d370-41d0-a5cc-692a8469fac9 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:51.312600 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 534/1051] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:51.318098 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59202110-5535-484c-8f31-c0640a7f68be None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:51.412384 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:51.421644 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59202110-5535-484c-8f31-c0640a7f68be None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:51.454896 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec48185-19ef-4493-b7c9-d36746a46f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:51.455529 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec48185-19ef-4493-b7c9-d36746a46f11 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:51.455811 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec48185-19ef-4493-b7c9-d36746a46f11 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:51.456107 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec48185-19ef-4493-b7c9-d36746a46f11 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:51.463840 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec48185-19ef-4493-b7c9-d36746a46f11 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:51.465313 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec48185-19ef-4493-b7c9-d36746a46f11 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:51.476545 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-8ec48185-19ef-4493-b7c9-d36746a46f11 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 15:13:51.476545 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:51.476545 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:51.476545 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:51.476545 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:51.476545 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:51.476545 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:51.476545 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:51.476545 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:51.476545 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:51.476545 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:51.476545 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:51.476545 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 15:13:51.476545 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 15:13:51.476545 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 15:13:51.476545 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 15:13:51.477540 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:51.477540 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:51.477540 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 15:13:51.477540 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:51.477540 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 15:13:51.477540 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:51.477540 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:51.477540 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:51.477540 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:51.477540 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:51.477540 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:51.477540 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:51.477540 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:51.477540 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:51.477540 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:51.477540 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:51.477540 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:51.477540 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:51.478485 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:51.478485 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:51.478485 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 15:13:51.478485 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 15:13:51.478485 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 15:13:51.478485 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 15:13:51.478485 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 15:13:51.478485 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 15:13:51.478485 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 15:13:51.478485 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:51.478485 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 518/1052] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Aug 5 15:13:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:51.485551 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acb776d-d370-41d0-a5cc-692a8469fac9 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:51.558287 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:51.589851 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acb776d-d370-41d0-a5cc-692a8469fac9 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:51.628008 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1171fa-ad92-46ab-a85d-b833d65fe755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:51.628663 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1171fa-ad92-46ab-a85d-b833d65fe755 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:51.628949 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1171fa-ad92-46ab-a85d-b833d65fe755 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:51.629283 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1171fa-ad92-46ab-a85d-b833d65fe755 None admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:51.630094 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1171fa-ad92-46ab-a85d-b833d65fe755 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:51.630408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1171fa-ad92-46ab-a85d-b833d65fe755 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:51.631840 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1171fa-ad92-46ab-a85d-b833d65fe755 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:51.654334 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 535/1053] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Aug 5 15:13:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:51.656806 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec48185-19ef-4493-b7c9-d36746a46f11 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:51.761545 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec48185-19ef-4493-b7c9-d36746a46f11 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:51.797317 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13749f63-7dae-45d6-9a4e-48f54d93acac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:51.797978 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13749f63-7dae-45d6-9a4e-48f54d93acac None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:51.798259 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13749f63-7dae-45d6-9a4e-48f54d93acac None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:51.798533 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13749f63-7dae-45d6-9a4e-48f54d93acac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:51.810184 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13749f63-7dae-45d6-9a4e-48f54d93acac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:51.811692 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13749f63-7dae-45d6-9a4e-48f54d93acac None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:51.830730 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-13749f63-7dae-45d6-9a4e-48f54d93acac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:51.830730 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:51.830730 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:51.830730 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:51.830730 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:51.830730 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:51.830730 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:51.830730 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:51.830730 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:51.830730 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:51.830730 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:51.830730 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:51.830730 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:51.830730 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:51.830730 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:51.831721 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:51.831721 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:51.831721 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:51.831721 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:51.831721 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:51.831721 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:51.831721 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:51.831721 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:51.831721 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:51.831721 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:51.831721 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:51.831721 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:51.831721 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:51.831721 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:51.831721 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:51.831721 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:51.831721 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:51.832826 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:51.832826 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:51.832826 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:51.832826 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:51.832826 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 519/1054] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:51.839432 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1171fa-ad92-46ab-a85d-b833d65fe755 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:51.944425 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1171fa-ad92-46ab-a85d-b833d65fe755 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:51.974374 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:51.980888 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8999f4ee-00cf-4dc0-b289-b93474204e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:51.982022 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8999f4ee-00cf-4dc0-b289-b93474204e03 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:51.982491 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8999f4ee-00cf-4dc0-b289-b93474204e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:51.983356 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8999f4ee-00cf-4dc0-b289-b93474204e03 None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:51.984080 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8999f4ee-00cf-4dc0-b289-b93474204e03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:51.985090 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8999f4ee-00cf-4dc0-b289-b93474204e03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:51.987074 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8999f4ee-00cf-4dc0-b289-b93474204e03 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:52.007461 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 536/1055] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:52.010992 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13749f63-7dae-45d6-9a4e-48f54d93acac None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:52.012884 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:52.123287 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13749f63-7dae-45d6-9a4e-48f54d93acac None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:52.163297 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a9322-1cf5-42eb-9ddf-e0c095133558 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:52.164186 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a9322-1cf5-42eb-9ddf-e0c095133558 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:52.164186 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a9322-1cf5-42eb-9ddf-e0c095133558 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:52.165620 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 537/1056] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 15:13:52.170079 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de4c7f4-e49f-4a12-b152-b8c078ae5802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:52.170079 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4c7f4-e49f-4a12-b152-b8c078ae5802 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:52.170079 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4c7f4-e49f-4a12-b152-b8c078ae5802 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:52.170079 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4c7f4-e49f-4a12-b152-b8c078ae5802 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:52.175175 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2bfa54-4c04-4afa-aedf-01fb90370239 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:52.175465 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2bfa54-4c04-4afa-aedf-01fb90370239 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:52.175777 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2bfa54-4c04-4afa-aedf-01fb90370239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:52.181720 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de4c7f4-e49f-4a12-b152-b8c078ae5802 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:52.183163 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de4c7f4-e49f-4a12-b152-b8c078ae5802 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:52.198271 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-4de4c7f4-e49f-4a12-b152-b8c078ae5802 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:52.198271 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:52.198271 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:52.198271 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:52.198271 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:52.198271 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:52.198271 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:52.198271 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:52.198271 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:52.198271 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:52.198271 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:52.198271 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:52.198271 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 15:13:52.198271 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 15:13:52.198271 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 15:13:52.199208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 15:13:52.199208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:52.199208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:52.199208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:52.199208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:52.199208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:52.199208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:52.199208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:52.199208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:52.199208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:52.199208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:52.199208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:52.199208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:52.199208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:52.199208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:52.199208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:52.199208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:52.200153 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 15:13:52.200153 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 15:13:52.200153 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 15:13:52.200153 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 15:13:52.200153 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 15:13:52.200153 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 15:13:52.200153 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:52.200153 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:52.200153 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 520/1057] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:13:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:13:52.206382 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de4c7f4-e49f-4a12-b152-b8c078ae5802 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:52.207187 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-3a2bfa54-4c04-4afa-aedf-01fb90370239 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:52.208815 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-3a2bfa54-4c04-4afa-aedf-01fb90370239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:52.283728 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:52.291893 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 538/1058] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:52 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 15:13:52.314711 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2bfa54-4c04-4afa-aedf-01fb90370239 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:52.324753 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de4c7f4-e49f-4a12-b152-b8c078ae5802 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:52.361551 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b8bb2-33ff-4955-98a4-9c008e8ddfcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:52.362183 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b8bb2-33ff-4955-98a4-9c008e8ddfcc None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:52.362470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b8bb2-33ff-4955-98a4-9c008e8ddfcc None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:52.362747 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b8bb2-33ff-4955-98a4-9c008e8ddfcc None admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:52.363518 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b8bb2-33ff-4955-98a4-9c008e8ddfcc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:52.363816 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b8bb2-33ff-4955-98a4-9c008e8ddfcc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:52.365281 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b8bb2-33ff-4955-98a4-9c008e8ddfcc None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:52.378423 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 521/1059] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 15:13:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:52.384390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b8bb2-33ff-4955-98a4-9c008e8ddfcc None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:52.424932 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2bfa54-4c04-4afa-aedf-01fb90370239 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:52.446484 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:52.463062 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb40fa-2c0f-46e0-a3ad-163ee4b4a120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:52.463062 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb40fa-2c0f-46e0-a3ad-163ee4b4a120 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:52.463062 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb40fa-2c0f-46e0-a3ad-163ee4b4a120 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:52.463376 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb40fa-2c0f-46e0-a3ad-163ee4b4a120 None admin] PATH_INFO: `/v3/services/image` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:52.464182 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeb40fa-2c0f-46e0-a3ad-163ee4b4a120 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:52.465864 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeb40fa-2c0f-46e0-a3ad-163ee4b4a120 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:52.475387 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-ceeb40fa-2c0f-46e0-a3ad-163ee4b4a120 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 15:13:52.475387 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:52.475387 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 15:13:52.475387 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 15:13:52.475387 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 15:13:52.475387 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 15:13:52.475387 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 15:13:52.475387 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:52.475387 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 15:13:52.475387 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:52.475387 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 15:13:52.475387 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:52.475387 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:52.475387 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:52.475387 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:52.475387 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:52.475387 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:52.480475 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:52.480475 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:52.480475 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:52.480475 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:52.480475 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:52.480475 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:52.480475 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 15:13:52.480475 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 15:13:52.480475 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 15:13:52.480475 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 15:13:52.480475 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:52.480475 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:52.480475 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:52.480475 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:52.480475 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:52.480475 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:52.482198 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:52.482198 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:52.482198 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:52.482198 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:52.482198 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:52.482198 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:52.482198 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:52.482198 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:52.482198 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:52.482198 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:52.482198 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 15:13:52.482198 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:13:52.482198 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 15:13:52.482198 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:52.482198 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 539/1060] 10.222.0.40 () {56 vars in 1105 bytes} [Fri Aug 5 15:13:52 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:13:52.483824 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb40fa-2c0f-46e0-a3ad-163ee4b4a120 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:52.495512 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b8bb2-33ff-4955-98a4-9c008e8ddfcc None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:52.522053 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:52.532843 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5bbaee-80b4-4a89-8597-d72a70da9829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:52.533515 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bbaee-80b4-4a89-8597-d72a70da9829 None admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:52.533982 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bbaee-80b4-4a89-8597-d72a70da9829 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:52.534442 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bbaee-80b4-4a89-8597-d72a70da9829 None admin] PATH_INFO: `/v3/projects/06f27ab8886a4c0d81c4d4e3fa100a83/users/bc0497dab1f44bb7b8196d39bf01d770/roles/e7638910be904dc388b948d044a90758` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:52.536495 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5bbaee-80b4-4a89-8597-d72a70da9829 None admin] RBAC: Authorizing `identity:create_grant(project_id=06f27ab8886a4c0d81c4d4e3fa100a83, user_id=bc0497dab1f44bb7b8196d39bf01d770, role_id=e7638910be904dc388b948d044a90758)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:52.537926 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5bbaee-80b4-4a89-8597-d72a70da9829 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:52.587131 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb40fa-2c0f-46e0-a3ad-163ee4b4a120 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:52.620532 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d0ab76-f5ca-40a2-8c16-b01c0a816c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:52.621148 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0ab76-f5ca-40a2-8c16-b01c0a816c10 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:52.621700 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0ab76-f5ca-40a2-8c16-b01c0a816c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:52.621980 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0ab76-f5ca-40a2-8c16-b01c0a816c10 None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:52.622658 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d0ab76-f5ca-40a2-8c16-b01c0a816c10 None admin] RBAC: Adding query filter params (name=image) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:52.622969 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d0ab76-f5ca-40a2-8c16-b01c0a816c10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:52.624932 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d0ab76-f5ca-40a2-8c16-b01c0a816c10 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:52.629730 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 540/1061] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:52 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:52.635500 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d0ab76-f5ca-40a2-8c16-b01c0a816c10 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:52.652454 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 522/1062] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 15:13:52 2022] PUT /identity/v3/projects/06f27ab8886a4c0d81c4d4e3fa100a83/users/bc0497dab1f44bb7b8196d39bf01d770/roles/e7638910be904dc388b948d044a90758 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:13:52.692237 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:52.749439 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d0ab76-f5ca-40a2-8c16-b01c0a816c10 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:52.788141 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a260316-fa3a-4b6c-bca5-742999c5ce73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:52.788780 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a260316-fa3a-4b6c-bca5-742999c5ce73 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:52.789066 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a260316-fa3a-4b6c-bca5-742999c5ce73 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:52.789344 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a260316-fa3a-4b6c-bca5-742999c5ce73 None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:52.790035 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a260316-fa3a-4b6c-bca5-742999c5ce73 None admin] RBAC: Adding query filter params (type=image) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:52.790345 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a260316-fa3a-4b6c-bca5-742999c5ce73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:52.791722 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a260316-fa3a-4b6c-bca5-742999c5ce73 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:52.800012 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 541/1063] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:52 2022] GET /identity/v3/services?type=image => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:52.805978 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5bbaee-80b4-4a89-8597-d72a70da9829 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:52.956375 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5bbaee-80b4-4a89-8597-d72a70da9829 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:53.011056 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a9cc2-525b-4253-a136-6808cb938346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:53.012054 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a9cc2-525b-4253-a136-6808cb938346 None admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:53.012557 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a9cc2-525b-4253-a136-6808cb938346 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:53.013034 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a9cc2-525b-4253-a136-6808cb938346 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:53.014292 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873a9cc2-525b-4253-a136-6808cb938346 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:53.016732 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873a9cc2-525b-4253-a136-6808cb938346 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:53.133145 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 523/1064] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:13:52 2022] POST /identity/v3/endpoints => generated 327 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:13:53.209484 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242158b-6181-496f-997e-50b209e28066 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:53.209841 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242158b-6181-496f-997e-50b209e28066 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:53.210049 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242158b-6181-496f-997e-50b209e28066 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:53.211662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 542/1065] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:53.217873 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91d3d4-4c31-40d2-b770-e6acc30576a8 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:53.218190 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91d3d4-4c31-40d2-b770-e6acc30576a8 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:53.218441 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91d3d4-4c31-40d2-b770-e6acc30576a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:53.238594 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-9f91d3d4-4c31-40d2-b770-e6acc30576a8 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:53.239837 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-9f91d3d4-4c31-40d2-b770-e6acc30576a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:53.280390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:53.322555 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 524/1066] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:13:53.346163 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4242158b-6181-496f-997e-50b209e28066 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:53.454225 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4242158b-6181-496f-997e-50b209e28066 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:53.467782 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:53.489703 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eee45d-2604-41b8-ac07-8ede8676050c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:53.490353 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eee45d-2604-41b8-ac07-8ede8676050c None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:53.490638 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eee45d-2604-41b8-ac07-8ede8676050c None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:53.490915 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eee45d-2604-41b8-ac07-8ede8676050c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:53.502564 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eee45d-2604-41b8-ac07-8ede8676050c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:13:53.507494 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-10eee45d-2604-41b8-ac07-8ede8676050c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:13:53.507494 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:53.507494 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:53.507494 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:53.507494 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:53.507494 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:53.507494 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:53.507494 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:53.507494 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:53.507494 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:53.507494 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:53.507494 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:53.507494 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:13:53.507494 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:13:53.507494 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:13:53.507494 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:13:53.508464 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:13:53.508464 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:13:53.508464 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:13:53.508464 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:13:53.508464 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:53.508464 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:53.508464 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:53.508464 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:53.508464 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:53.508464 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:53.508464 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:53.508464 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:53.508464 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:53.508464 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:53.508464 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:53.508464 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:53.509325 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:53.509325 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:53.509325 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:53.509325 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:53.509325 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:13:53.509325 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:13:53.509325 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:13:53.509325 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:13:53.509325 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:13:53.509325 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:13:53.509325 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:13:53.509325 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:53.509325 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 543/1067] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 15:13:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:53.516532 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f91d3d4-4c31-40d2-b770-e6acc30576a8 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:53.619441 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f91d3d4-4c31-40d2-b770-e6acc30576a8 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:53.656356 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb50beb-2fd1-4115-83ef-26803203d6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:53.656984 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb50beb-2fd1-4115-83ef-26803203d6f8 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:53.657272 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb50beb-2fd1-4115-83ef-26803203d6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:53.657554 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb50beb-2fd1-4115-83ef-26803203d6f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:53.658270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb50beb-2fd1-4115-83ef-26803203d6f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:53.658583 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb50beb-2fd1-4115-83ef-26803203d6f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:53.659957 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb50beb-2fd1-4115-83ef-26803203d6f8 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:53.668399 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 525/1068] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:13:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:53.674033 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eee45d-2604-41b8-ac07-8ede8676050c None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:53.785829 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eee45d-2604-41b8-ac07-8ede8676050c None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:53.828466 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59509386-2ca1-4af4-820b-edf7dbd52efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:53.829442 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59509386-2ca1-4af4-820b-edf7dbd52efc None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:53.829924 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59509386-2ca1-4af4-820b-edf7dbd52efc None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:53.830385 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59509386-2ca1-4af4-820b-edf7dbd52efc None admin] PATH_INFO: `/v3/users/4627b01d391b4d99a9ef9d9ed92731c0` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:53.846289 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59509386-2ca1-4af4-820b-edf7dbd52efc None admin] RBAC: Authorizing `identity:get_user(user_id=4627b01d391b4d99a9ef9d9ed92731c0)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:53.847737 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59509386-2ca1-4af4-820b-edf7dbd52efc None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:53.854054 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 544/1069] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:13:53 2022] GET /identity/v3/users/4627b01d391b4d99a9ef9d9ed92731c0 => generated 285 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:53.855331 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb50beb-2fd1-4115-83ef-26803203d6f8 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:53.886441 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:53.960094 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb50beb-2fd1-4115-83ef-26803203d6f8 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:53.993325 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66df7eb-a118-4281-90cb-b8e979af3ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:53.993942 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66df7eb-a118-4281-90cb-b8e979af3ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:53.994230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66df7eb-a118-4281-90cb-b8e979af3ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:53.994508 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66df7eb-a118-4281-90cb-b8e979af3ad0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:54.062758 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66df7eb-a118-4281-90cb-b8e979af3ad0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:54.064243 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66df7eb-a118-4281-90cb-b8e979af3ad0 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:54.090887 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:54.170643 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 526/1070] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:13:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 15:13:54.172391 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59509386-2ca1-4af4-820b-edf7dbd52efc None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:54.207906 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:54.281161 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59509386-2ca1-4af4-820b-edf7dbd52efc None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:54.317637 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9554826d-d323-46d9-bf3a-6e27482fab84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:54.318284 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554826d-d323-46d9-bf3a-6e27482fab84 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:54.318563 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554826d-d323-46d9-bf3a-6e27482fab84 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:54.318837 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554826d-d323-46d9-bf3a-6e27482fab84 None admin] PATH_INFO: `/v3/projects/789f5a4a3c6f4d8cb42ea3cb270b5eb3` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:54.320021 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9554826d-d323-46d9-bf3a-6e27482fab84 None admin] RBAC: Authorizing `identity:get_project(project_id=789f5a4a3c6f4d8cb42ea3cb270b5eb3)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:54.321471 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9554826d-d323-46d9-bf3a-6e27482fab84 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:54.328154 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 545/1071] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:13:54 2022] GET /identity/v3/projects/789f5a4a3c6f4d8cb42ea3cb270b5eb3 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:54.329212 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66df7eb-a118-4281-90cb-b8e979af3ad0 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:54.427389 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66df7eb-a118-4281-90cb-b8e979af3ad0 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:54.453643 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:54.461921 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0547190-bcf0-4a6a-af7b-8c1a6eb0c531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:54.462537 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0547190-bcf0-4a6a-af7b-8c1a6eb0c531 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:54.462817 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0547190-bcf0-4a6a-af7b-8c1a6eb0c531 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:54.463091 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0547190-bcf0-4a6a-af7b-8c1a6eb0c531 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:54.463875 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0547190-bcf0-4a6a-af7b-8c1a6eb0c531 None admin] RBAC: Adding query filter params (role.id=7c88c815db854954b516ab97cd471647, scope.project.id=789f5a4a3c6f4d8cb42ea3cb270b5eb3, user.id=4627b01d391b4d99a9ef9d9ed92731c0) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:54.464215 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0547190-bcf0-4a6a-af7b-8c1a6eb0c531 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:54.465631 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0547190-bcf0-4a6a-af7b-8c1a6eb0c531 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:54.476398 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 527/1072] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:13:54 2022] GET /identity/v3/role_assignments?user.id=4627b01d391b4d99a9ef9d9ed92731c0&scope.project.id=789f5a4a3c6f4d8cb42ea3cb270b5eb3&role.id=7c88c815db854954b516ab97cd471647 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:54.682669 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4058e-d142-403b-b10c-cd0f5373d253 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:54.683155 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4058e-d142-403b-b10c-cd0f5373d253 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:54.683155 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4058e-d142-403b-b10c-cd0f5373d253 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:54.684691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 546/1073] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:54.689880 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ceab6d-1dc4-41ba-aa79-5e4bc93ba8f8 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:54.690173 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ceab6d-1dc4-41ba-aa79-5e4bc93ba8f8 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:54.690453 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ceab6d-1dc4-41ba-aa79-5e4bc93ba8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:54.708523 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-b3ceab6d-1dc4-41ba-aa79-5e4bc93ba8f8 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:54.709721 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-b3ceab6d-1dc4-41ba-aa79-5e4bc93ba8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:54.777776 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 528/1074] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:13:54.801745 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee4058e-d142-403b-b10c-cd0f5373d253 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:54.901512 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:54.907470 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee4058e-d142-403b-b10c-cd0f5373d253 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:54.942392 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae3c268-ffbf-4a2d-bf42-164d20cf1bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:54.943301 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae3c268-ffbf-4a2d-bf42-164d20cf1bf9 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:54.943765 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae3c268-ffbf-4a2d-bf42-164d20cf1bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:54.944250 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae3c268-ffbf-4a2d-bf42-164d20cf1bf9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:54.954715 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae3c268-ffbf-4a2d-bf42-164d20cf1bf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:13:54.959958 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-2ae3c268-ffbf-4a2d-bf42-164d20cf1bf9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 15:13:54.959958 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:54.959958 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:54.959958 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:54.959958 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:54.959958 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:54.959958 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:54.959958 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:54.959958 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:54.959958 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:54.959958 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:54.959958 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:54.959958 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:13:54.959958 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:13:54.959958 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:13:54.959958 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:13:54.961063 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:13:54.961063 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:13:54.961063 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:13:54.961063 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:13:54.961063 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:54.961063 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:54.961063 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:54.961063 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:54.961063 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:54.961063 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:54.961063 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:54.961063 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:54.961063 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:54.961063 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:54.961063 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:54.961063 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:54.962136 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:54.962136 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:54.962136 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:54.962136 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:54.962136 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:13:54.962136 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:13:54.962136 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:13:54.962136 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:13:54.962136 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:13:54.962136 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:13:54.962136 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 15:13:54.962136 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:54.964272 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 547/1075] 10.222.0.40 () {56 vars in 1120 bytes} [Fri Aug 5 15:13:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:54.970906 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ceab6d-1dc4-41ba-aa79-5e4bc93ba8f8 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:55.072220 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ceab6d-1dc4-41ba-aa79-5e4bc93ba8f8 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:55.096125 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:55.108153 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b1a6a-74dd-4e8b-acea-44a82ecfaca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:55.108783 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b1a6a-74dd-4e8b-acea-44a82ecfaca4 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:55.109071 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b1a6a-74dd-4e8b-acea-44a82ecfaca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:55.109353 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b1a6a-74dd-4e8b-acea-44a82ecfaca4 None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:55.110061 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8b1a6a-74dd-4e8b-acea-44a82ecfaca4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:55.110372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8b1a6a-74dd-4e8b-acea-44a82ecfaca4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:55.111734 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8b1a6a-74dd-4e8b-acea-44a82ecfaca4 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:55.124514 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 529/1076] 10.222.0.40 () {56 vars in 1115 bytes} [Fri Aug 5 15:13:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:55.125987 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae3c268-ffbf-4a2d-bf42-164d20cf1bf9 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:55.227923 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc9eb3-f97e-4b58-b309-449a1503d3d0 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:55.228189 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc9eb3-f97e-4b58-b309-449a1503d3d0 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:55.228443 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc9eb3-f97e-4b58-b309-449a1503d3d0 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:55.230071 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 530/1077] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:55.234797 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc3444-3edb-48a4-93d3-23cf1dbdd63f None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:55.235090 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc3444-3edb-48a4-93d3-23cf1dbdd63f None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:55.235391 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc3444-3edb-48a4-93d3-23cf1dbdd63f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:55.242227 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae3c268-ffbf-4a2d-bf42-164d20cf1bf9 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:55.253271 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-51dc3444-3edb-48a4-93d3-23cf1dbdd63f None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:55.254540 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-51dc3444-3edb-48a4-93d3-23cf1dbdd63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:55.279732 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e392bc5f-5b1a-489e-9211-8d31301a36a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:55.280406 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392bc5f-5b1a-489e-9211-8d31301a36a0 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:55.280692 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392bc5f-5b1a-489e-9211-8d31301a36a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:55.280968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392bc5f-5b1a-489e-9211-8d31301a36a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:55.291772 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e392bc5f-5b1a-489e-9211-8d31301a36a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:55.293247 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e392bc5f-5b1a-489e-9211-8d31301a36a0 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:55.307702 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-e392bc5f-5b1a-489e-9211-8d31301a36a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:55.307702 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:55.307702 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:55.307702 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:55.307702 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:55.307702 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:55.307702 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:55.307702 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:55.307702 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:55.307702 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:55.307702 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:55.307702 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:55.307702 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:55.307702 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:55.307702 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:55.308691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:55.308691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:55.308691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:55.308691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:55.308691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:55.308691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:55.308691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:55.308691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:55.308691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:55.308691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:55.308691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:55.308691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:55.308691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:55.308691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:55.308691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:55.308691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:55.308691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:55.309607 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:55.309607 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:55.309607 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:55.309607 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:55.309607 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 548/1078] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:55.315665 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e392bc5f-5b1a-489e-9211-8d31301a36a0 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:55.328887 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:55.341068 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 531/1079] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 15:13:55.363488 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc3444-3edb-48a4-93d3-23cf1dbdd63f None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:55.422856 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e392bc5f-5b1a-489e-9211-8d31301a36a0 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:55.463949 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786eb12b-daa3-4b24-b5e8-782011d1e31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:55.464738 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786eb12b-daa3-4b24-b5e8-782011d1e31e None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:55.465028 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786eb12b-daa3-4b24-b5e8-782011d1e31e None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:55.465314 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786eb12b-daa3-4b24-b5e8-782011d1e31e None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:55.466016 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786eb12b-daa3-4b24-b5e8-782011d1e31e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:55.466332 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786eb12b-daa3-4b24-b5e8-782011d1e31e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:55.467707 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786eb12b-daa3-4b24-b5e8-782011d1e31e None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:55.478364 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc3444-3edb-48a4-93d3-23cf1dbdd63f None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:55.484709 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 549/1080] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:55.485407 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786eb12b-daa3-4b24-b5e8-782011d1e31e None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:55.494725 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:55.517909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511820ff-24cb-4f82-861c-259c0d044c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:55.518534 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511820ff-24cb-4f82-861c-259c0d044c3f None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:55.518822 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511820ff-24cb-4f82-861c-259c0d044c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:55.519102 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511820ff-24cb-4f82-861c-259c0d044c3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:55.566549 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:55.594591 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511820ff-24cb-4f82-861c-259c0d044c3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:55.596056 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511820ff-24cb-4f82-861c-259c0d044c3f None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:55.598256 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786eb12b-daa3-4b24-b5e8-782011d1e31e None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:55.633745 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e1855-212d-4ca5-8dc8-91405121cff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:55.634394 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e1855-212d-4ca5-8dc8-91405121cff1 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:55.634683 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e1855-212d-4ca5-8dc8-91405121cff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:55.634965 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e1855-212d-4ca5-8dc8-91405121cff1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:55.642842 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e1855-212d-4ca5-8dc8-91405121cff1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:55.644334 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e1855-212d-4ca5-8dc8-91405121cff1 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:55.655915 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-cc8e1855-212d-4ca5-8dc8-91405121cff1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 15:13:55.655915 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:55.655915 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:55.655915 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:55.655915 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:55.655915 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:55.655915 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:55.655915 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:55.655915 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:55.655915 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:55.655915 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:55.655915 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:55.655915 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 15:13:55.655915 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 15:13:55.655915 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 15:13:55.655915 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 15:13:55.656957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:55.656957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:55.656957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 15:13:55.656957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:55.656957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 15:13:55.656957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:13:55.656957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:55.656957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:55.656957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:55.656957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:55.656957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:55.656957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:55.656957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:55.656957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:55.656957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:55.656957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:55.656957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:55.656957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:55.657926 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:55.657926 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:55.657926 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 15:13:55.657926 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 15:13:55.657926 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 15:13:55.657926 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 15:13:55.657926 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 15:13:55.657926 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 15:13:55.657926 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 15:13:55.657926 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:55.657926 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 550/1081] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Aug 5 15:13:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:13:55.663123 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e1855-212d-4ca5-8dc8-91405121cff1 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:55.736515 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 532/1082] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:13:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 05 15:13:55.737543 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511820ff-24cb-4f82-861c-259c0d044c3f None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:55.754487 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:55.768504 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e1855-212d-4ca5-8dc8-91405121cff1 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:55.798727 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:55.804375 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ca4f8-5c0b-4991-a01d-5c387598c8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:55.804998 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ca4f8-5c0b-4991-a01d-5c387598c8ba None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:55.805283 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ca4f8-5c0b-4991-a01d-5c387598c8ba None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:55.805650 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ca4f8-5c0b-4991-a01d-5c387598c8ba None admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:55.806447 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4ca4f8-5c0b-4991-a01d-5c387598c8ba None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:55.806791 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4ca4f8-5c0b-4991-a01d-5c387598c8ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:55.808223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4ca4f8-5c0b-4991-a01d-5c387598c8ba None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:55.827241 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 551/1083] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Aug 5 15:13:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:55.832900 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ca4f8-5c0b-4991-a01d-5c387598c8ba None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:55.852702 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511820ff-24cb-4f82-861c-259c0d044c3f None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:55.890289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512fc922-61e9-45ab-a3e3-77cce5e0abe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:55.890939 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fc922-61e9-45ab-a3e3-77cce5e0abe4 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:55.891228 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fc922-61e9-45ab-a3e3-77cce5e0abe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:55.891521 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fc922-61e9-45ab-a3e3-77cce5e0abe4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:55.901782 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512fc922-61e9-45ab-a3e3-77cce5e0abe4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:55.903229 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512fc922-61e9-45ab-a3e3-77cce5e0abe4 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:55.918470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-512fc922-61e9-45ab-a3e3-77cce5e0abe4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:55.918470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:55.918470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:55.918470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:55.918470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:55.918470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:55.918470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:55.918470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:55.918470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:55.918470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:55.918470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:55.918470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:55.918470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:55.918470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:55.918470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:55.919571 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:55.919571 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:55.919571 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:55.919571 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:55.919571 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:55.919571 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:55.919571 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:55.919571 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:55.919571 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:55.919571 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:55.919571 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:55.919571 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:55.919571 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:55.919571 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:55.919571 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:55.919571 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:55.919571 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:55.920770 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:55.920770 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:55.920770 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:55.920770 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:55.924757 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 533/1084] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:55.925465 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512fc922-61e9-45ab-a3e3-77cce5e0abe4 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:55.945799 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ca4f8-5c0b-4991-a01d-5c387598c8ba None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:55.983546 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44696f8-6884-40ec-8a36-f003bcf08262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:55.984207 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44696f8-6884-40ec-8a36-f003bcf08262 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:55.984545 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44696f8-6884-40ec-8a36-f003bcf08262 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:55.984778 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44696f8-6884-40ec-8a36-f003bcf08262 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:55.998808 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:56.037969 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512fc922-61e9-45ab-a3e3-77cce5e0abe4 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:56.045445 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:56.058964 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44696f8-6884-40ec-8a36-f003bcf08262 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:56.060493 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44696f8-6884-40ec-8a36-f003bcf08262 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:56.074412 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d13329-2a28-4c80-bcdd-2c2e75fd0fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:56.075621 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d13329-2a28-4c80-bcdd-2c2e75fd0fdc None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:56.076754 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d13329-2a28-4c80-bcdd-2c2e75fd0fdc None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:56.077041 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d13329-2a28-4c80-bcdd-2c2e75fd0fdc None admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:56.077746 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d13329-2a28-4c80-bcdd-2c2e75fd0fdc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:56.078068 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d13329-2a28-4c80-bcdd-2c2e75fd0fdc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:56.079492 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d13329-2a28-4c80-bcdd-2c2e75fd0fdc None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:56.092510 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 534/1085] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:56.191724 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 552/1086] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:13:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 05 15:13:56.197403 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d13329-2a28-4c80-bcdd-2c2e75fd0fdc None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:56.307772 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d13329-2a28-4c80-bcdd-2c2e75fd0fdc None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:56.337134 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:56.346625 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e4b042-ca48-4b1b-b1d7-2351cfea20ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:56.349764 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4b042-ca48-4b1b-b1d7-2351cfea20ae None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:56.349764 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4b042-ca48-4b1b-b1d7-2351cfea20ae None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:56.349764 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4b042-ca48-4b1b-b1d7-2351cfea20ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:56.358971 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e4b042-ca48-4b1b-b1d7-2351cfea20ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:56.360461 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e4b042-ca48-4b1b-b1d7-2351cfea20ae None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:56.377042 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-28e4b042-ca48-4b1b-b1d7-2351cfea20ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:56.377042 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:56.377042 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:56.377042 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:56.377042 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:56.377042 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:56.377042 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:56.377042 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:56.377042 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:56.377042 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:56.377042 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:56.377042 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:56.377042 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:13:56.377042 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:13:56.377042 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:13:56.377996 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:13:56.377996 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:56.377996 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:56.377996 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:56.377996 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:56.377996 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:56.377996 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:56.377996 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:56.377996 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:56.377996 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:56.377996 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:56.377996 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:56.377996 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:56.377996 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:56.377996 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:56.377996 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:56.377996 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:56.378917 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:13:56.378917 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:13:56.378917 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:13:56.378917 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:56.378917 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 535/1087] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:13:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:56.383996 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44696f8-6884-40ec-8a36-f003bcf08262 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:56.455214 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31633ac9-5713-4c09-ae24-8763959cd34e None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:56.455501 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31633ac9-5713-4c09-ae24-8763959cd34e None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:56.455785 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31633ac9-5713-4c09-ae24-8763959cd34e None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:56.457191 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 536/1088] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 15:13:56.462100 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad6c459-7ec6-4d30-bf11-6ced204f0e53 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:56.462387 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad6c459-7ec6-4d30-bf11-6ced204f0e53 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:56.462670 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad6c459-7ec6-4d30-bf11-6ced204f0e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:56.472274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:56.480488 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-aad6c459-7ec6-4d30-bf11-6ced204f0e53 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:56.481685 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-aad6c459-7ec6-4d30-bf11-6ced204f0e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:56.489593 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44696f8-6884-40ec-8a36-f003bcf08262 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:56.519797 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683464ef-c97f-48e9-9497-0258741a51cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:56.520450 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683464ef-c97f-48e9-9497-0258741a51cd None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:56.520741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683464ef-c97f-48e9-9497-0258741a51cd None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:56.521017 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683464ef-c97f-48e9-9497-0258741a51cd None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:56.526674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683464ef-c97f-48e9-9497-0258741a51cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:56.526674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683464ef-c97f-48e9-9497-0258741a51cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:56.526674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683464ef-c97f-48e9-9497-0258741a51cd None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:56.539292 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 553/1089] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:13:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:56.540328 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683464ef-c97f-48e9-9497-0258741a51cd None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:56.550211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 537/1090] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 15:13:56.572879 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad6c459-7ec6-4d30-bf11-6ced204f0e53 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:56.640727 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683464ef-c97f-48e9-9497-0258741a51cd None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:56.668400 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad6c459-7ec6-4d30-bf11-6ced204f0e53 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:56.673318 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd6ceb-5419-4cd7-bc15-fd4f2cbe42c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:56.673953 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd6ceb-5419-4cd7-bc15-fd4f2cbe42c0 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:56.674238 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd6ceb-5419-4cd7-bc15-fd4f2cbe42c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:56.674513 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd6ceb-5419-4cd7-bc15-fd4f2cbe42c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:56.678064 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:56.681086 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccd6ceb-5419-4cd7-bc15-fd4f2cbe42c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:56.682538 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccd6ceb-5419-4cd7-bc15-fd4f2cbe42c0 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:56.694458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-9ccd6ceb-5419-4cd7-bc15-fd4f2cbe42c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:56.694458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:56.694458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:56.694458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:56.694458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:56.694458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:56.694458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:56.694458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:56.694458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:56.694458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:56.694458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:56.694458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:56.694458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 15:13:56.694458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 15:13:56.694458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 15:13:56.695503 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 15:13:56.695503 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:56.695503 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:56.695503 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:56.695503 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:56.695503 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:56.695503 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:56.695503 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:56.695503 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:56.695503 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:56.695503 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:56.695503 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:56.695503 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:56.695503 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:56.695503 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:56.695503 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:56.695503 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:56.696725 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 15:13:56.696725 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 15:13:56.696725 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 15:13:56.696725 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 15:13:56.696725 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 15:13:56.696725 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 15:13:56.696725 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:13:56.696725 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:56.696725 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 554/1091] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:13:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:56.701494 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd6ceb-5419-4cd7-bc15-fd4f2cbe42c0 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:56.705227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab9cb5-7538-4e83-956f-9d619dedc7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:56.705872 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab9cb5-7538-4e83-956f-9d619dedc7dd None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:56.709804 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab9cb5-7538-4e83-956f-9d619dedc7dd None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:56.709804 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab9cb5-7538-4e83-956f-9d619dedc7dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:56.712297 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:56.716331 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dab9cb5-7538-4e83-956f-9d619dedc7dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:13:56.721141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-1dab9cb5-7538-4e83-956f-9d619dedc7dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:13:56.721141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:56.721141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:56.721141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:56.721141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:56.721141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:56.721141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:56.721141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:56.721141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:56.721141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:56.721141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:56.721141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:56.721141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:13:56.721141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:13:56.721141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:13:56.721141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:13:56.722268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:13:56.722268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:13:56.722268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:13:56.722268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:13:56.722268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:56.722268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:56.722268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:56.722268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:56.722268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:56.722268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:56.722268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:56.722268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:56.722268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:56.722268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:56.722268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:56.722268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:56.723242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:56.723242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:56.723242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:56.723242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:56.723242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:13:56.723242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:13:56.723242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:13:56.723242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:13:56.723242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:13:56.723242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:13:56.723242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:13:56.723242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:13:56.724389 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 538/1092] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 15:13:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:13:56.728352 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab9cb5-7538-4e83-956f-9d619dedc7dd None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:56.801543 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd6ceb-5419-4cd7-bc15-fd4f2cbe42c0 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:56.826811 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab9cb5-7538-4e83-956f-9d619dedc7dd None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:56.835678 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2d691-f27f-4c49-acf5-cfe094946bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:56.836326 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2d691-f27f-4c49-acf5-cfe094946bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:56.836618 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2d691-f27f-4c49-acf5-cfe094946bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:56.836898 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2d691-f27f-4c49-acf5-cfe094946bb2 None admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:56.837680 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac2d691-f27f-4c49-acf5-cfe094946bb2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:56.837994 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac2d691-f27f-4c49-acf5-cfe094946bb2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:56.839528 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac2d691-f27f-4c49-acf5-cfe094946bb2 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:56.856674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 555/1093] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 15:13:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:56.858203 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2d691-f27f-4c49-acf5-cfe094946bb2 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:56.859773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94779019-c2d9-4048-91fd-d09c1aa24ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:56.860418 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94779019-c2d9-4048-91fd-d09c1aa24ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:56.860709 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94779019-c2d9-4048-91fd-d09c1aa24ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:56.860981 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94779019-c2d9-4048-91fd-d09c1aa24ea4 None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:56.861688 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94779019-c2d9-4048-91fd-d09c1aa24ea4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:56.861990 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94779019-c2d9-4048-91fd-d09c1aa24ea4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:56.863327 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94779019-c2d9-4048-91fd-d09c1aa24ea4 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:56.872028 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 539/1094] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:13:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:56.878456 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94779019-c2d9-4048-91fd-d09c1aa24ea4 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:56.907513 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:56.923581 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:56.955702 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2d691-f27f-4c49-acf5-cfe094946bb2 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:56.975163 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94779019-c2d9-4048-91fd-d09c1aa24ea4 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:56.990815 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0646ef28-b577-47c1-afd8-3ec137d0e398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:56.991431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646ef28-b577-47c1-afd8-3ec137d0e398 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:56.991722 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646ef28-b577-47c1-afd8-3ec137d0e398 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:56.992004 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646ef28-b577-47c1-afd8-3ec137d0e398 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:56.992827 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0646ef28-b577-47c1-afd8-3ec137d0e398 None admin] RBAC: Adding query filter params (role.id=e7638910be904dc388b948d044a90758, scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83, user.id=bc0497dab1f44bb7b8196d39bf01d770) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:56.993142 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0646ef28-b577-47c1-afd8-3ec137d0e398 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:56.994564 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0646ef28-b577-47c1-afd8-3ec137d0e398 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:57.014603 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 556/1095] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:13:56 2022] GET /identity/v3/role_assignments?user.id=bc0497dab1f44bb7b8196d39bf01d770&scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83&role.id=e7638910be904dc388b948d044a90758 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:57.014818 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7776f487-4369-435b-9903-8f64f85b54c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:57.014818 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7776f487-4369-435b-9903-8f64f85b54c1 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:57.014818 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7776f487-4369-435b-9903-8f64f85b54c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:57.014818 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7776f487-4369-435b-9903-8f64f85b54c1 None admin] PATH_INFO: `/v3/users/4627b01d391b4d99a9ef9d9ed92731c0` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:57.015536 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7776f487-4369-435b-9903-8f64f85b54c1 None admin] RBAC: Authorizing `identity:get_user(user_id=4627b01d391b4d99a9ef9d9ed92731c0)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:57.015536 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7776f487-4369-435b-9903-8f64f85b54c1 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:57.015536 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 540/1096] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:13:56 2022] GET /identity/v3/users/4627b01d391b4d99a9ef9d9ed92731c0 => generated 285 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:13:57.020660 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0646ef28-b577-47c1-afd8-3ec137d0e398 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:57.125877 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0646ef28-b577-47c1-afd8-3ec137d0e398 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:57.155933 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:57.161211 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af69266-4372-406a-b38a-8ff5fbe1b2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:57.161857 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af69266-4372-406a-b38a-8ff5fbe1b2ef None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:57.167655 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af69266-4372-406a-b38a-8ff5fbe1b2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:57.167655 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af69266-4372-406a-b38a-8ff5fbe1b2ef None admin] PATH_INFO: `/v3/projects/789f5a4a3c6f4d8cb42ea3cb270b5eb3` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:57.167655 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af69266-4372-406a-b38a-8ff5fbe1b2ef None admin] RBAC: Authorizing `identity:get_project(project_id=789f5a4a3c6f4d8cb42ea3cb270b5eb3)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:57.167655 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af69266-4372-406a-b38a-8ff5fbe1b2ef None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:57.167655 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 557/1097] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:13:57 2022] GET /identity/v3/projects/789f5a4a3c6f4d8cb42ea3cb270b5eb3 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:57.173328 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7776f487-4369-435b-9903-8f64f85b54c1 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:57.275847 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:57.280279 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7776f487-4369-435b-9903-8f64f85b54c1 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:57.315609 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f58a6ec-68d5-4304-88f7-f3e8029ee21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:57.316307 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f58a6ec-68d5-4304-88f7-f3e8029ee21a None admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:57.316592 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f58a6ec-68d5-4304-88f7-f3e8029ee21a None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:57.316868 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f58a6ec-68d5-4304-88f7-f3e8029ee21a None admin] PATH_INFO: `/v3/projects/789f5a4a3c6f4d8cb42ea3cb270b5eb3/users/4627b01d391b4d99a9ef9d9ed92731c0/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:57.318827 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f58a6ec-68d5-4304-88f7-f3e8029ee21a None admin] RBAC: Authorizing `identity:create_grant(project_id=789f5a4a3c6f4d8cb42ea3cb270b5eb3, user_id=4627b01d391b4d99a9ef9d9ed92731c0, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:57.320283 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f58a6ec-68d5-4304-88f7-f3e8029ee21a None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:57.372519 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 541/1098] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 15:13:57 2022] PUT /identity/v3/projects/789f5a4a3c6f4d8cb42ea3cb270b5eb3/users/4627b01d391b4d99a9ef9d9ed92731c0/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:13:59.366487 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd3b93-fa07-48e9-8825-1f32309833a0 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:59.367004 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd3b93-fa07-48e9-8825-1f32309833a0 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:59.367123 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd3b93-fa07-48e9-8825-1f32309833a0 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:59.368441 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 558/1099] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:13:59.374402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8bf70-bcdd-4ca6-99c7-6236d4d2e08e None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:59.374804 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8bf70-bcdd-4ca6-99c7-6236d4d2e08e None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:59.374975 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8bf70-bcdd-4ca6-99c7-6236d4d2e08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:59.393119 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-87a8bf70-bcdd-4ca6-99c7-6236d4d2e08e None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:13:59.394298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-87a8bf70-bcdd-4ca6-99c7-6236d4d2e08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:13:59.459008 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 542/1100] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:13:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:13:59.482102 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd3b93-fa07-48e9-8825-1f32309833a0 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:59.629578 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd3b93-fa07-48e9-8825-1f32309833a0 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:59.661394 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874c0f9-eaee-4d51-a57a-afa1e0d5174b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:59.662042 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874c0f9-eaee-4d51-a57a-afa1e0d5174b None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:59.662317 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874c0f9-eaee-4d51-a57a-afa1e0d5174b None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:59.662589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874c0f9-eaee-4d51-a57a-afa1e0d5174b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:59.673280 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3874c0f9-eaee-4d51-a57a-afa1e0d5174b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:13:59.678101 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-3874c0f9-eaee-4d51-a57a-afa1e0d5174b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:13:59.678101 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:13:59.678101 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:13:59.678101 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:13:59.678101 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:13:59.678101 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:13:59.678101 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:13:59.678101 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:13:59.678101 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:13:59.678101 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:13:59.678101 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:13:59.678101 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:13:59.678101 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:13:59.678101 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:13:59.678101 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:13:59.678101 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:13:59.679245 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:13:59.679245 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:13:59.679245 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:13:59.679245 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:13:59.679245 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:13:59.679245 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:13:59.679245 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:13:59.679245 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:13:59.679245 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:13:59.679245 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:13:59.679245 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:13:59.679245 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:13:59.679245 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:13:59.679245 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:13:59.679245 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:13:59.679245 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:13:59.680565 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:13:59.680565 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:13:59.680565 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:13:59.680565 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:13:59.680565 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:13:59.680565 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:13:59.680565 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:13:59.680565 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:13:59.680565 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:13:59.680565 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:13:59.680565 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:13:59.680565 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:13:59.680565 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 559/1101] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 15:13:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:13:59.685223 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a8bf70-bcdd-4ca6-99c7-6236d4d2e08e None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:59.789891 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:59.802084 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a8bf70-bcdd-4ca6-99c7-6236d4d2e08e None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:59.835088 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d36c045-dd28-4516-a810-8c4a7f8d5668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:59.835692 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36c045-dd28-4516-a810-8c4a7f8d5668 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:59.835971 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36c045-dd28-4516-a810-8c4a7f8d5668 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:59.836271 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36c045-dd28-4516-a810-8c4a7f8d5668 None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:59.836977 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d36c045-dd28-4516-a810-8c4a7f8d5668 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:13:59.837283 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d36c045-dd28-4516-a810-8c4a7f8d5668 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:59.838637 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d36c045-dd28-4516-a810-8c4a7f8d5668 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:59.846560 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 543/1102] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:13:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:59.852407 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874c0f9-eaee-4d51-a57a-afa1e0d5174b None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:13:59.910462 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:13:59.949732 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874c0f9-eaee-4d51-a57a-afa1e0d5174b None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:13:59.981925 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e311f2-5335-48a7-83c1-31d6605c52df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:13:59.982550 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e311f2-5335-48a7-83c1-31d6605c52df None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:13:59.982836 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e311f2-5335-48a7-83c1-31d6605c52df None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:13:59.983681 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e311f2-5335-48a7-83c1-31d6605c52df None admin] PATH_INFO: `/v3/users/4627b01d391b4d99a9ef9d9ed92731c0` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:13:59.984907 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e311f2-5335-48a7-83c1-31d6605c52df None admin] RBAC: Authorizing `identity:get_user(user_id=4627b01d391b4d99a9ef9d9ed92731c0)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:13:59.986321 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e311f2-5335-48a7-83c1-31d6605c52df None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:13:59.987895 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 560/1103] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:13:59 2022] GET /identity/v3/users/4627b01d391b4d99a9ef9d9ed92731c0 => generated 285 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:13:59.993523 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d36c045-dd28-4516-a810-8c4a7f8d5668 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:00.088717 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d36c045-dd28-4516-a810-8c4a7f8d5668 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:00.123570 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9612c2-122f-4eb7-bd45-d467e37eb64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:00.124230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9612c2-122f-4eb7-bd45-d467e37eb64d None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:00.124513 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9612c2-122f-4eb7-bd45-d467e37eb64d None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:00.124791 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9612c2-122f-4eb7-bd45-d467e37eb64d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:00.142214 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:00.190404 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9612c2-122f-4eb7-bd45-d467e37eb64d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:00.191878 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9612c2-122f-4eb7-bd45-d467e37eb64d None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:00.288326 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 544/1104] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:13:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 15:14:00.294250 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e311f2-5335-48a7-83c1-31d6605c52df None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:00.391467 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e311f2-5335-48a7-83c1-31d6605c52df None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:00.416855 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:00.424346 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d377d70-18cf-42e0-9490-811e387e0bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:00.424976 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d377d70-18cf-42e0-9490-811e387e0bc6 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:00.425262 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d377d70-18cf-42e0-9490-811e387e0bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:00.425542 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d377d70-18cf-42e0-9490-811e387e0bc6 None admin] PATH_INFO: `/v3/projects/789f5a4a3c6f4d8cb42ea3cb270b5eb3` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:00.426689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d377d70-18cf-42e0-9490-811e387e0bc6 None admin] RBAC: Authorizing `identity:get_project(project_id=789f5a4a3c6f4d8cb42ea3cb270b5eb3)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:00.428137 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d377d70-18cf-42e0-9490-811e387e0bc6 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:00.429801 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 561/1105] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:14:00 2022] GET /identity/v3/projects/789f5a4a3c6f4d8cb42ea3cb270b5eb3 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:14:00.435801 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9612c2-122f-4eb7-bd45-d467e37eb64d None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:00.483893 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:00.532168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9612c2-122f-4eb7-bd45-d467e37eb64d None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:00.565206 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bf31ef-5be7-4437-af44-5887082a0ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:00.565798 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf31ef-5be7-4437-af44-5887082a0ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:00.566076 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf31ef-5be7-4437-af44-5887082a0ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:00.566351 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf31ef-5be7-4437-af44-5887082a0ef5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:00.567120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bf31ef-5be7-4437-af44-5887082a0ef5 None admin] RBAC: Adding query filter params (role.id=7c88c815db854954b516ab97cd471647, scope.project.id=789f5a4a3c6f4d8cb42ea3cb270b5eb3, user.id=4627b01d391b4d99a9ef9d9ed92731c0) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:14:00.567428 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bf31ef-5be7-4437-af44-5887082a0ef5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:00.568853 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bf31ef-5be7-4437-af44-5887082a0ef5 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:00.574358 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 545/1106] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:14:00 2022] GET /identity/v3/role_assignments?user.id=4627b01d391b4d99a9ef9d9ed92731c0&scope.project.id=789f5a4a3c6f4d8cb42ea3cb270b5eb3&role.id=7c88c815db854954b516ab97cd471647 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:14:02.575358 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7dd1d-5b3f-463d-8650-ecc46cb9aaa7 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:02.575906 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7dd1d-5b3f-463d-8650-ecc46cb9aaa7 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:02.575906 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7dd1d-5b3f-463d-8650-ecc46cb9aaa7 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:02.577304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 562/1107] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:14:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:14:02.582401 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561b328-267e-49ca-b6b0-276307b3aade None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:02.582800 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561b328-267e-49ca-b6b0-276307b3aade None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:02.582951 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561b328-267e-49ca-b6b0-276307b3aade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:02.600508 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-9561b328-267e-49ca-b6b0-276307b3aade None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:14:02.601689 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-9561b328-267e-49ca-b6b0-276307b3aade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:14:02.665375 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 546/1108] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:14:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:14:02.690247 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7dd1d-5b3f-463d-8650-ecc46cb9aaa7 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:02.797335 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7dd1d-5b3f-463d-8650-ecc46cb9aaa7 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:02.831378 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c7734d-319c-4ff9-8008-92e0785588e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:02.831940 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c7734d-319c-4ff9-8008-92e0785588e3 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:02.832265 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c7734d-319c-4ff9-8008-92e0785588e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:02.832541 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c7734d-319c-4ff9-8008-92e0785588e3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:02.833681 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c7734d-319c-4ff9-8008-92e0785588e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:02.835101 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c7734d-319c-4ff9-8008-92e0785588e3 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:02.836708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 563/1109] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:14:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:14:02.842521 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9561b328-267e-49ca-b6b0-276307b3aade None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:02.886677 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:02.939236 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9561b328-267e-49ca-b6b0-276307b3aade None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:02.971831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d62f61-d470-47aa-8281-549a5c217768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:02.972481 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d62f61-d470-47aa-8281-549a5c217768 None admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:02.972770 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d62f61-d470-47aa-8281-549a5c217768 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:02.973050 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d62f61-d470-47aa-8281-549a5c217768 None admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:02.973768 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d62f61-d470-47aa-8281-549a5c217768 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:02.975178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d62f61-d470-47aa-8281-549a5c217768 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:03.067799 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 547/1110] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 5 15:14:02 2022] POST /identity/v3/users => generated 286 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:14:05.090552 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2afef-c0c8-4c2f-9221-b43a96673a8e None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:05.091106 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2afef-c0c8-4c2f-9221-b43a96673a8e None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:05.091202 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2afef-c0c8-4c2f-9221-b43a96673a8e None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:05.093164 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 564/1111] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:14:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:14:05.098471 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6dc555-23dc-41c1-8247-2833d8d295d8 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:05.098872 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6dc555-23dc-41c1-8247-2833d8d295d8 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:05.099019 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6dc555-23dc-41c1-8247-2833d8d295d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:05.116584 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-1a6dc555-23dc-41c1-8247-2833d8d295d8 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:14:05.117758 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-1a6dc555-23dc-41c1-8247-2833d8d295d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:14:05.182888 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 548/1112] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:14:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:14:05.206840 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c2afef-c0c8-4c2f-9221-b43a96673a8e None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:05.268908 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:05.306122 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c2afef-c0c8-4c2f-9221-b43a96673a8e None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:05.339623 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e70dd0-9f31-4b4c-9eab-5b936958cae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:05.340288 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e70dd0-9f31-4b4c-9eab-5b936958cae4 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:05.340625 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e70dd0-9f31-4b4c-9eab-5b936958cae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:05.340861 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e70dd0-9f31-4b4c-9eab-5b936958cae4 None admin] PATH_INFO: `/v3/roles/944427ca14ee4c81a05ec67e54f50e7f` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:05.342336 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e70dd0-9f31-4b4c-9eab-5b936958cae4 None admin] RBAC: Authorizing `identity:get_role(role_id=944427ca14ee4c81a05ec67e54f50e7f)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:05.343702 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e70dd0-9f31-4b4c-9eab-5b936958cae4 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:05.345046 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 565/1113] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:14:05 2022] GET /identity/v3/roles/944427ca14ee4c81a05ec67e54f50e7f => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:14:05.350726 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6dc555-23dc-41c1-8247-2833d8d295d8 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:05.380006 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:05.447207 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6dc555-23dc-41c1-8247-2833d8d295d8 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:05.480426 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528cccbd-7e09-49ca-a71c-16f3bbddc50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:05.481055 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528cccbd-7e09-49ca-a71c-16f3bbddc50f None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:05.481339 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528cccbd-7e09-49ca-a71c-16f3bbddc50f None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:05.481615 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528cccbd-7e09-49ca-a71c-16f3bbddc50f None admin] PATH_INFO: `/v3/users/0995795b08e34d5c9188b6e54d3d26ca` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:05.499840 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528cccbd-7e09-49ca-a71c-16f3bbddc50f None admin] RBAC: Authorizing `identity:get_user(user_id=0995795b08e34d5c9188b6e54d3d26ca)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:05.501295 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528cccbd-7e09-49ca-a71c-16f3bbddc50f None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:05.502903 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 549/1114] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:14:05 2022] GET /identity/v3/users/0995795b08e34d5c9188b6e54d3d26ca => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:14:05.508595 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e70dd0-9f31-4b4c-9eab-5b936958cae4 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:05.604640 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e70dd0-9f31-4b4c-9eab-5b936958cae4 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:05.633178 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:05.637895 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84561bfb-9300-4290-9f47-391ae0c4a0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:05.638496 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84561bfb-9300-4290-9f47-391ae0c4a0fd None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:05.638779 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84561bfb-9300-4290-9f47-391ae0c4a0fd None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:05.639080 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84561bfb-9300-4290-9f47-391ae0c4a0fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:05.704070 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84561bfb-9300-4290-9f47-391ae0c4a0fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:05.705549 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84561bfb-9300-4290-9f47-391ae0c4a0fd None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:05.799830 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 566/1115] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:14:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 15:14:05.805578 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528cccbd-7e09-49ca-a71c-16f3bbddc50f None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:05.891574 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:05.903165 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528cccbd-7e09-49ca-a71c-16f3bbddc50f None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:05.936818 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e2548-419f-4dd5-a672-bdc3fc231567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:05.937450 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e2548-419f-4dd5-a672-bdc3fc231567 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:05.937735 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e2548-419f-4dd5-a672-bdc3fc231567 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:05.938013 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e2548-419f-4dd5-a672-bdc3fc231567 None admin] PATH_INFO: `/v3/projects/789f5a4a3c6f4d8cb42ea3cb270b5eb3` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:05.939162 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161e2548-419f-4dd5-a672-bdc3fc231567 None admin] RBAC: Authorizing `identity:get_project(project_id=789f5a4a3c6f4d8cb42ea3cb270b5eb3)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:05.940614 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161e2548-419f-4dd5-a672-bdc3fc231567 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:05.942264 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 550/1116] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:14:05 2022] GET /identity/v3/projects/789f5a4a3c6f4d8cb42ea3cb270b5eb3 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:14:05.948328 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84561bfb-9300-4290-9f47-391ae0c4a0fd None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:05.978228 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:06.044480 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84561bfb-9300-4290-9f47-391ae0c4a0fd None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:06.076798 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098adb3-4d8b-437d-946c-3b2d5d8cf916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:06.077392 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098adb3-4d8b-437d-946c-3b2d5d8cf916 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:06.077674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098adb3-4d8b-437d-946c-3b2d5d8cf916 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:06.077947 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098adb3-4d8b-437d-946c-3b2d5d8cf916 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:06.078721 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f098adb3-4d8b-437d-946c-3b2d5d8cf916 None admin] RBAC: Adding query filter params (role.id=944427ca14ee4c81a05ec67e54f50e7f, scope.project.id=789f5a4a3c6f4d8cb42ea3cb270b5eb3, user.id=0995795b08e34d5c9188b6e54d3d26ca) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:14:06.079030 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f098adb3-4d8b-437d-946c-3b2d5d8cf916 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:06.080484 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f098adb3-4d8b-437d-946c-3b2d5d8cf916 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:06.085790 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 567/1117] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:14:05 2022] GET /identity/v3/role_assignments?user.id=0995795b08e34d5c9188b6e54d3d26ca&scope.project.id=789f5a4a3c6f4d8cb42ea3cb270b5eb3&role.id=944427ca14ee4c81a05ec67e54f50e7f => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:14:08.037725 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb62553-27a9-41cf-b32a-b5a40037c94c None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:08.038258 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb62553-27a9-41cf-b32a-b5a40037c94c None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:08.038258 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb62553-27a9-41cf-b32a-b5a40037c94c None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:08.039610 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 551/1118] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:14:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:14:08.044705 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1b6e3-055e-4f22-88e4-4ac9c526c79b None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:08.045108 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1b6e3-055e-4f22-88e4-4ac9c526c79b None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:08.045318 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1b6e3-055e-4f22-88e4-4ac9c526c79b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:08.063042 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-65e1b6e3-055e-4f22-88e4-4ac9c526c79b None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:14:08.064265 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-65e1b6e3-055e-4f22-88e4-4ac9c526c79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:14:08.129865 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 568/1119] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:14:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:14:08.152470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb62553-27a9-41cf-b32a-b5a40037c94c None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:08.252331 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb62553-27a9-41cf-b32a-b5a40037c94c None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:08.286829 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe9880d-87a0-4f54-9378-e1a3746883e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:08.287467 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9880d-87a0-4f54-9378-e1a3746883e2 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:08.287804 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9880d-87a0-4f54-9378-e1a3746883e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:08.288037 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9880d-87a0-4f54-9378-e1a3746883e2 None admin] PATH_INFO: `/v3/roles/944427ca14ee4c81a05ec67e54f50e7f` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:08.289537 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe9880d-87a0-4f54-9378-e1a3746883e2 None admin] RBAC: Authorizing `identity:get_role(role_id=944427ca14ee4c81a05ec67e54f50e7f)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:08.290898 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe9880d-87a0-4f54-9378-e1a3746883e2 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:08.292285 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 552/1120] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:14:08 2022] GET /identity/v3/roles/944427ca14ee4c81a05ec67e54f50e7f => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:14:08.298127 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e1b6e3-055e-4f22-88e4-4ac9c526c79b None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:08.325367 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:08.396825 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e1b6e3-055e-4f22-88e4-4ac9c526c79b None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:08.429189 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c70c2-cfb1-4ddb-8423-0f84be919208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:08.429824 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c70c2-cfb1-4ddb-8423-0f84be919208 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:08.430115 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c70c2-cfb1-4ddb-8423-0f84be919208 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:08.430402 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c70c2-cfb1-4ddb-8423-0f84be919208 None admin] PATH_INFO: `/v3/users/0995795b08e34d5c9188b6e54d3d26ca` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:08.431567 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708c70c2-cfb1-4ddb-8423-0f84be919208 None admin] RBAC: Authorizing `identity:get_user(user_id=0995795b08e34d5c9188b6e54d3d26ca)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:08.433024 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708c70c2-cfb1-4ddb-8423-0f84be919208 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:08.434629 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 569/1121] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:14:08 2022] GET /identity/v3/users/0995795b08e34d5c9188b6e54d3d26ca => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:14:08.440847 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe9880d-87a0-4f54-9378-e1a3746883e2 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:08.465486 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:08.537155 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe9880d-87a0-4f54-9378-e1a3746883e2 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:08.571207 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991825dd-a83a-4192-b827-42d4ad406fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:08.571781 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991825dd-a83a-4192-b827-42d4ad406fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:08.572091 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991825dd-a83a-4192-b827-42d4ad406fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:08.572375 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991825dd-a83a-4192-b827-42d4ad406fd4 None admin] PATH_INFO: `/v3/projects/789f5a4a3c6f4d8cb42ea3cb270b5eb3` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:08.573533 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991825dd-a83a-4192-b827-42d4ad406fd4 None admin] RBAC: Authorizing `identity:get_project(project_id=789f5a4a3c6f4d8cb42ea3cb270b5eb3)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:08.574946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991825dd-a83a-4192-b827-42d4ad406fd4 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:08.576634 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 553/1122] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:14:08 2022] GET /identity/v3/projects/789f5a4a3c6f4d8cb42ea3cb270b5eb3 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:14:08.582462 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c70c2-cfb1-4ddb-8423-0f84be919208 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:08.672118 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:08.678390 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c70c2-cfb1-4ddb-8423-0f84be919208 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:08.711599 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c033b6e-d189-4d18-81f3-55bc22dd28d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:08.712276 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c033b6e-d189-4d18-81f3-55bc22dd28d6 None admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:08.712557 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c033b6e-d189-4d18-81f3-55bc22dd28d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:08.712834 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c033b6e-d189-4d18-81f3-55bc22dd28d6 None admin] PATH_INFO: `/v3/projects/789f5a4a3c6f4d8cb42ea3cb270b5eb3/users/0995795b08e34d5c9188b6e54d3d26ca/roles/944427ca14ee4c81a05ec67e54f50e7f` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:08.714972 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c033b6e-d189-4d18-81f3-55bc22dd28d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=789f5a4a3c6f4d8cb42ea3cb270b5eb3, user_id=0995795b08e34d5c9188b6e54d3d26ca, role_id=944427ca14ee4c81a05ec67e54f50e7f)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:08.716412 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c033b6e-d189-4d18-81f3-55bc22dd28d6 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:08.770241 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 570/1123] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 15:14:08 2022] PUT /identity/v3/projects/789f5a4a3c6f4d8cb42ea3cb270b5eb3/users/0995795b08e34d5c9188b6e54d3d26ca/roles/944427ca14ee4c81a05ec67e54f50e7f => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:14:10.744233 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992cf5b8-ddb8-49cd-8491-5243102833b6 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:10.744769 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992cf5b8-ddb8-49cd-8491-5243102833b6 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:10.744769 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992cf5b8-ddb8-49cd-8491-5243102833b6 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:10.746411 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 554/1124] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:14:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:14:10.754029 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921064d-4c07-45d3-a981-a6ab6cf7fd04 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:10.754669 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921064d-4c07-45d3-a981-a6ab6cf7fd04 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:10.754669 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921064d-4c07-45d3-a981-a6ab6cf7fd04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:10.774246 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-8921064d-4c07-45d3-a981-a6ab6cf7fd04 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:14:10.775421 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-8921064d-4c07-45d3-a981-a6ab6cf7fd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:14:10.839960 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 571/1125] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:14:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:14:10.863348 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992cf5b8-ddb8-49cd-8491-5243102833b6 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:10.954589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:10.965010 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992cf5b8-ddb8-49cd-8491-5243102833b6 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:10.999309 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfb0360-1d9f-47aa-9daa-b63f91e85138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:10.999965 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfb0360-1d9f-47aa-9daa-b63f91e85138 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:11.000276 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfb0360-1d9f-47aa-9daa-b63f91e85138 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:11.000554 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfb0360-1d9f-47aa-9daa-b63f91e85138 None admin] PATH_INFO: `/v3/roles/944427ca14ee4c81a05ec67e54f50e7f` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:11.001974 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfb0360-1d9f-47aa-9daa-b63f91e85138 None admin] RBAC: Authorizing `identity:get_role(role_id=944427ca14ee4c81a05ec67e54f50e7f)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:11.003356 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfb0360-1d9f-47aa-9daa-b63f91e85138 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:11.004745 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 555/1126] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:14:10 2022] GET /identity/v3/roles/944427ca14ee4c81a05ec67e54f50e7f => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:14:11.010548 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8921064d-4c07-45d3-a981-a6ab6cf7fd04 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:11.090940 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:11.105984 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8921064d-4c07-45d3-a981-a6ab6cf7fd04 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:11.140394 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b1ff26-06df-47a6-9f2a-467fce87a855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:11.141026 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1ff26-06df-47a6-9f2a-467fce87a855 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:11.141312 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1ff26-06df-47a6-9f2a-467fce87a855 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:11.141591 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1ff26-06df-47a6-9f2a-467fce87a855 None admin] PATH_INFO: `/v3/users/0995795b08e34d5c9188b6e54d3d26ca` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:11.142750 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b1ff26-06df-47a6-9f2a-467fce87a855 None admin] RBAC: Authorizing `identity:get_user(user_id=0995795b08e34d5c9188b6e54d3d26ca)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:11.144210 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b1ff26-06df-47a6-9f2a-467fce87a855 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:11.145788 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 572/1127] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:14:11 2022] GET /identity/v3/users/0995795b08e34d5c9188b6e54d3d26ca => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:14:11.151372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfb0360-1d9f-47aa-9daa-b63f91e85138 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:11.248383 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfb0360-1d9f-47aa-9daa-b63f91e85138 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:11.280946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e824e4e-6872-4386-9e84-de00407a7a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:11.281548 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e824e4e-6872-4386-9e84-de00407a7a4a None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:11.281834 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e824e4e-6872-4386-9e84-de00407a7a4a None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:11.282112 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e824e4e-6872-4386-9e84-de00407a7a4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:11.308602 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:11.346976 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e824e4e-6872-4386-9e84-de00407a7a4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:11.348427 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e824e4e-6872-4386-9e84-de00407a7a4a None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:11.442505 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 556/1128] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:14:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 15:14:11.448322 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b1ff26-06df-47a6-9f2a-467fce87a855 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:11.544950 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b1ff26-06df-47a6-9f2a-467fce87a855 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:11.577474 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208066b-8f83-45da-99d1-2a004435332c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:11.578097 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208066b-8f83-45da-99d1-2a004435332c None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:11.578383 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208066b-8f83-45da-99d1-2a004435332c None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:11.578663 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208066b-8f83-45da-99d1-2a004435332c None admin] PATH_INFO: `/v3/projects/789f5a4a3c6f4d8cb42ea3cb270b5eb3` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:11.579809 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d208066b-8f83-45da-99d1-2a004435332c None admin] RBAC: Authorizing `identity:get_project(project_id=789f5a4a3c6f4d8cb42ea3cb270b5eb3)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:11.581262 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d208066b-8f83-45da-99d1-2a004435332c None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:11.582896 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 573/1129] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:14:11 2022] GET /identity/v3/projects/789f5a4a3c6f4d8cb42ea3cb270b5eb3 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:14:11.588668 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e824e4e-6872-4386-9e84-de00407a7a4a None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:11.645024 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:11.683159 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e824e4e-6872-4386-9e84-de00407a7a4a None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:11.717203 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ee2e6-cc2d-4a15-9c52-ce71e8d949ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:11.717802 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ee2e6-cc2d-4a15-9c52-ce71e8d949ad None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:11.718084 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ee2e6-cc2d-4a15-9c52-ce71e8d949ad None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:11.718358 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ee2e6-cc2d-4a15-9c52-ce71e8d949ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:11.719143 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3ee2e6-cc2d-4a15-9c52-ce71e8d949ad None admin] RBAC: Adding query filter params (role.id=944427ca14ee4c81a05ec67e54f50e7f, scope.project.id=789f5a4a3c6f4d8cb42ea3cb270b5eb3, user.id=0995795b08e34d5c9188b6e54d3d26ca) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:14:11.719456 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3ee2e6-cc2d-4a15-9c52-ce71e8d949ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:11.720880 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3ee2e6-cc2d-4a15-9c52-ce71e8d949ad None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:11.726402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 557/1130] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:14:11 2022] GET /identity/v3/role_assignments?user.id=0995795b08e34d5c9188b6e54d3d26ca&scope.project.id=789f5a4a3c6f4d8cb42ea3cb270b5eb3&role.id=944427ca14ee4c81a05ec67e54f50e7f => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:14:13.728414 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc73df8-3c46-4671-b90b-b6eca213aea7 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:13.728961 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc73df8-3c46-4671-b90b-b6eca213aea7 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:13.728961 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc73df8-3c46-4671-b90b-b6eca213aea7 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:13.730326 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 574/1131] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:14:13.735445 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486eed8e-a1fe-4340-8f86-8c416cc0efef None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:13.735851 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486eed8e-a1fe-4340-8f86-8c416cc0efef None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:13.736005 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486eed8e-a1fe-4340-8f86-8c416cc0efef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:13.754351 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-486eed8e-a1fe-4340-8f86-8c416cc0efef None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:14:13.755537 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-486eed8e-a1fe-4340-8f86-8c416cc0efef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:14:13.820355 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 558/1132] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:14:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 15:14:13.843015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc73df8-3c46-4671-b90b-b6eca213aea7 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:13.864978 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:13.942353 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc73df8-3c46-4671-b90b-b6eca213aea7 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:13.974366 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda97876-8b51-4fba-9536-21011a6694c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:13.974992 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda97876-8b51-4fba-9536-21011a6694c4 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:13.975276 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda97876-8b51-4fba-9536-21011a6694c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:13.975551 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda97876-8b51-4fba-9536-21011a6694c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:13.976694 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda97876-8b51-4fba-9536-21011a6694c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:13.978127 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda97876-8b51-4fba-9536-21011a6694c4 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:13.979741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 575/1133] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:14:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:14:13.985900 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486eed8e-a1fe-4340-8f86-8c416cc0efef None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:14.038616 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:14.081737 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486eed8e-a1fe-4340-8f86-8c416cc0efef None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:14.114025 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446af236-5dfb-4803-a96e-428819d10dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:14.114626 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446af236-5dfb-4803-a96e-428819d10dea None admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:14.114910 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446af236-5dfb-4803-a96e-428819d10dea None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:14.115183 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446af236-5dfb-4803-a96e-428819d10dea None admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:14.115843 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446af236-5dfb-4803-a96e-428819d10dea None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:14.117292 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446af236-5dfb-4803-a96e-428819d10dea None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:14.223806 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 559/1134] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 5 15:14:13 2022] POST /identity/v3/projects => generated 311 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:14:16.264913 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d98c5-379e-4d04-a72d-7c7f95401bee None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:16.265445 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d98c5-379e-4d04-a72d-7c7f95401bee None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:16.265445 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d98c5-379e-4d04-a72d-7c7f95401bee None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:16.266827 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 576/1135] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:14:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:14:16.272044 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe07ca2-82af-4a86-9a00-7fb1dc6c42ae None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:16.272504 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe07ca2-82af-4a86-9a00-7fb1dc6c42ae None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:16.272610 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe07ca2-82af-4a86-9a00-7fb1dc6c42ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:16.290303 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-cbe07ca2-82af-4a86-9a00-7fb1dc6c42ae None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:14:16.291476 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-cbe07ca2-82af-4a86-9a00-7fb1dc6c42ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:14:16.364677 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 560/1136] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:14:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:14:16.389846 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d98c5-379e-4d04-a72d-7c7f95401bee None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:16.481347 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:16.494359 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d98c5-379e-4d04-a72d-7c7f95401bee None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:16.529353 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ff6d8d-7fb2-411e-bec9-17a764852590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:16.530240 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff6d8d-7fb2-411e-bec9-17a764852590 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:16.530525 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff6d8d-7fb2-411e-bec9-17a764852590 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:16.530806 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff6d8d-7fb2-411e-bec9-17a764852590 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:16.531995 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ff6d8d-7fb2-411e-bec9-17a764852590 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:16.533462 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ff6d8d-7fb2-411e-bec9-17a764852590 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:16.535080 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 577/1137] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:14:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:14:16.541089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe07ca2-82af-4a86-9a00-7fb1dc6c42ae None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:16.576479 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:16.641148 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe07ca2-82af-4a86-9a00-7fb1dc6c42ae None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:16.676849 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676cf070-0484-47ff-a014-216c7c4d9a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:16.677466 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676cf070-0484-47ff-a014-216c7c4d9a1a None admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:16.677750 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676cf070-0484-47ff-a014-216c7c4d9a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:16.678026 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676cf070-0484-47ff-a014-216c7c4d9a1a None admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:16.678700 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676cf070-0484-47ff-a014-216c7c4d9a1a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:16.680173 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676cf070-0484-47ff-a014-216c7c4d9a1a None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:16.759339 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 561/1138] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 5 15:14:16 2022] POST /identity/v3/users => generated 286 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:14:18.815976 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21b35e-af63-46f6-86f3-46ee34bcfde0 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:18.816560 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21b35e-af63-46f6-86f3-46ee34bcfde0 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:18.816560 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21b35e-af63-46f6-86f3-46ee34bcfde0 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:18.817914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 578/1139] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:14:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:14:18.823040 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc052c9f-1b82-4dba-a27a-3bb0428302a7 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:18.823455 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc052c9f-1b82-4dba-a27a-3bb0428302a7 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:18.823604 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc052c9f-1b82-4dba-a27a-3bb0428302a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:18.841854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-cc052c9f-1b82-4dba-a27a-3bb0428302a7 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:14:18.843032 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-cc052c9f-1b82-4dba-a27a-3bb0428302a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:14:18.907350 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 562/1140] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:14:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:14:18.930366 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b21b35e-af63-46f6-86f3-46ee34bcfde0 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:19.030556 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b21b35e-af63-46f6-86f3-46ee34bcfde0 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:19.062963 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58736f-d5a6-4655-8a15-d5f07553717d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:19.063587 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58736f-d5a6-4655-8a15-d5f07553717d None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:19.063872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58736f-d5a6-4655-8a15-d5f07553717d None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:19.064180 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58736f-d5a6-4655-8a15-d5f07553717d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:19.074151 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c58736f-d5a6-4655-8a15-d5f07553717d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:14:19.079558 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-9c58736f-d5a6-4655-8a15-d5f07553717d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:14:19.079558 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:14:19.079558 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:14:19.079558 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:14:19.079558 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:14:19.079558 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:14:19.079558 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:14:19.079558 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:14:19.079558 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:14:19.079558 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:14:19.079558 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:14:19.079558 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:14:19.079558 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:14:19.079558 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:14:19.079558 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:14:19.079558 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:14:19.080741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:14:19.080741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:14:19.080741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:14:19.080741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:14:19.080741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:14:19.080741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:14:19.080741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:14:19.080741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:14:19.080741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:14:19.080741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:14:19.080741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:14:19.080741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:14:19.080741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:14:19.080741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:14:19.080741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:14:19.080741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:14:19.081770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:14:19.081770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:14:19.081770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:14:19.081770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:14:19.081770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:14:19.081770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:14:19.081770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:14:19.081770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:14:19.081770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:14:19.081770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:14:19.081770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:14:19.081770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:14:19.081770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 579/1141] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 15:14:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:14:19.086574 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc052c9f-1b82-4dba-a27a-3bb0428302a7 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:19.109838 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:19.181835 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc052c9f-1b82-4dba-a27a-3bb0428302a7 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:19.217056 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464e3f79-035d-4654-bbc0-17e30f5c39b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:19.217666 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e3f79-035d-4654-bbc0-17e30f5c39b6 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:19.217952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e3f79-035d-4654-bbc0-17e30f5c39b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:19.218230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e3f79-035d-4654-bbc0-17e30f5c39b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:19.218938 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464e3f79-035d-4654-bbc0-17e30f5c39b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:14:19.219246 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464e3f79-035d-4654-bbc0-17e30f5c39b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:19.220634 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464e3f79-035d-4654-bbc0-17e30f5c39b6 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:19.228662 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 563/1142] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:14:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:14:19.234376 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58736f-d5a6-4655-8a15-d5f07553717d None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:19.252875 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:19.333731 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58736f-d5a6-4655-8a15-d5f07553717d None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:19.366149 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1963fe-83e0-43c6-8fbd-3745bed26e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:19.366768 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1963fe-83e0-43c6-8fbd-3745bed26e90 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:19.367048 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1963fe-83e0-43c6-8fbd-3745bed26e90 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:19.367324 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1963fe-83e0-43c6-8fbd-3745bed26e90 None admin] PATH_INFO: `/v3/users/d7228b454f0146fe92b200a4b05ec0cb` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:19.380525 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1963fe-83e0-43c6-8fbd-3745bed26e90 None admin] RBAC: Authorizing `identity:get_user(user_id=d7228b454f0146fe92b200a4b05ec0cb)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:19.381948 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1963fe-83e0-43c6-8fbd-3745bed26e90 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:19.383530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 580/1143] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:14:19 2022] GET /identity/v3/users/d7228b454f0146fe92b200a4b05ec0cb => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:14:19.389338 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464e3f79-035d-4654-bbc0-17e30f5c39b6 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:19.471251 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:19.486987 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464e3f79-035d-4654-bbc0-17e30f5c39b6 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:19.519163 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171330e5-a241-4cee-bd14-56954cacc8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:19.519766 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171330e5-a241-4cee-bd14-56954cacc8fb None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:19.520050 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171330e5-a241-4cee-bd14-56954cacc8fb None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:19.520359 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171330e5-a241-4cee-bd14-56954cacc8fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:19.585726 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171330e5-a241-4cee-bd14-56954cacc8fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:19.587110 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171330e5-a241-4cee-bd14-56954cacc8fb None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:19.667026 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:19.683279 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 564/1144] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:14:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 15:14:19.689188 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1963fe-83e0-43c6-8fbd-3745bed26e90 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:19.787724 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:19.811633 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1963fe-83e0-43c6-8fbd-3745bed26e90 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:19.845729 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d552f969-228d-4e50-918f-aa8aefd57a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:19.846318 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552f969-228d-4e50-918f-aa8aefd57a54 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:19.846566 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552f969-228d-4e50-918f-aa8aefd57a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:19.846839 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552f969-228d-4e50-918f-aa8aefd57a54 None admin] PATH_INFO: `/v3/projects/a7209d04d43448508d2beea46f10a2e9` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:19.847978 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d552f969-228d-4e50-918f-aa8aefd57a54 None admin] RBAC: Authorizing `identity:get_project(project_id=a7209d04d43448508d2beea46f10a2e9)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:19.849424 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d552f969-228d-4e50-918f-aa8aefd57a54 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:19.851178 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 581/1145] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:14:19 2022] GET /identity/v3/projects/a7209d04d43448508d2beea46f10a2e9 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:14:19.857260 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171330e5-a241-4cee-bd14-56954cacc8fb None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:19.952599 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171330e5-a241-4cee-bd14-56954cacc8fb None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:19.985493 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded0111-1cd9-4838-8d50-c9bd1cb3648d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:19.986093 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded0111-1cd9-4838-8d50-c9bd1cb3648d None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:19.986379 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded0111-1cd9-4838-8d50-c9bd1cb3648d None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:19.986659 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded0111-1cd9-4838-8d50-c9bd1cb3648d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:19.987442 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded0111-1cd9-4838-8d50-c9bd1cb3648d None admin] RBAC: Adding query filter params (role.id=7c88c815db854954b516ab97cd471647, scope.project.id=a7209d04d43448508d2beea46f10a2e9, user.id=d7228b454f0146fe92b200a4b05ec0cb) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:14:19.987754 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded0111-1cd9-4838-8d50-c9bd1cb3648d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:19.989197 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded0111-1cd9-4838-8d50-c9bd1cb3648d None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:19.994358 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 565/1146] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:14:19 2022] GET /identity/v3/role_assignments?user.id=d7228b454f0146fe92b200a4b05ec0cb&scope.project.id=a7209d04d43448508d2beea46f10a2e9&role.id=7c88c815db854954b516ab97cd471647 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:14:21.961110 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d0916f-caf0-4f6d-8431-8b368a0b56d4 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:21.961666 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d0916f-caf0-4f6d-8431-8b368a0b56d4 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:21.961666 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d0916f-caf0-4f6d-8431-8b368a0b56d4 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:21.963006 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 582/1147] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:14:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:14:21.968233 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528e106-be7e-44fd-bcf8-79fff83e65bb None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:21.968606 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528e106-be7e-44fd-bcf8-79fff83e65bb None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:21.968800 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528e106-be7e-44fd-bcf8-79fff83e65bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:21.986435 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-c528e106-be7e-44fd-bcf8-79fff83e65bb None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:14:21.987624 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-c528e106-be7e-44fd-bcf8-79fff83e65bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:14:22.030880 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:22.053295 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 566/1148] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:14:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:14:22.076120 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d0916f-caf0-4f6d-8431-8b368a0b56d4 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:22.174309 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d0916f-caf0-4f6d-8431-8b368a0b56d4 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:22.207251 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c13e426-acbb-4fec-ab0f-23c41b30cd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:22.207875 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c13e426-acbb-4fec-ab0f-23c41b30cd46 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:22.208193 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c13e426-acbb-4fec-ab0f-23c41b30cd46 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:22.208470 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c13e426-acbb-4fec-ab0f-23c41b30cd46 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:22.218302 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c13e426-acbb-4fec-ab0f-23c41b30cd46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:14:22.223039 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-6c13e426-acbb-4fec-ab0f-23c41b30cd46 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:14:22.223039 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:14:22.223039 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:14:22.223039 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:14:22.223039 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:14:22.223039 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:14:22.223039 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:14:22.223039 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:14:22.223039 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:14:22.223039 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:14:22.223039 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:14:22.223039 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:14:22.223039 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:14:22.223039 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:14:22.223039 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:14:22.223039 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:14:22.224145 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:14:22.224145 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:14:22.224145 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:14:22.224145 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:14:22.224145 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:14:22.224145 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:14:22.224145 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:14:22.224145 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:14:22.224145 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:14:22.224145 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:14:22.224145 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:14:22.224145 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:14:22.224145 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:14:22.224145 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:14:22.224145 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:14:22.224145 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:14:22.225013 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:14:22.225013 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:14:22.225013 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:14:22.225013 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:14:22.225013 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:14:22.225013 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:14:22.225013 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:14:22.225013 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:14:22.225013 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:14:22.225013 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:14:22.225013 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:14:22.225013 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:14:22.225013 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 583/1149] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 15:14:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:14:22.230165 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c528e106-be7e-44fd-bcf8-79fff83e65bb None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:22.324927 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c528e106-be7e-44fd-bcf8-79fff83e65bb None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:22.358292 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2bb143-efc1-4d02-8fea-b2b7a43eb2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:22.358898 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2bb143-efc1-4d02-8fea-b2b7a43eb2b1 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:22.359177 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2bb143-efc1-4d02-8fea-b2b7a43eb2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:22.359449 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2bb143-efc1-4d02-8fea-b2b7a43eb2b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:22.360175 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2bb143-efc1-4d02-8fea-b2b7a43eb2b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:14:22.360482 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2bb143-efc1-4d02-8fea-b2b7a43eb2b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:22.361829 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2bb143-efc1-4d02-8fea-b2b7a43eb2b1 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:22.369663 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 567/1150] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:14:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:14:22.376400 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c13e426-acbb-4fec-ab0f-23c41b30cd46 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:22.379753 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:22.477524 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c13e426-acbb-4fec-ab0f-23c41b30cd46 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:22.510660 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b166a1-01aa-4fcb-af21-c4acb6615894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:22.511292 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b166a1-01aa-4fcb-af21-c4acb6615894 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:22.511578 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b166a1-01aa-4fcb-af21-c4acb6615894 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:22.511857 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b166a1-01aa-4fcb-af21-c4acb6615894 None admin] PATH_INFO: `/v3/users/d7228b454f0146fe92b200a4b05ec0cb` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:22.513042 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b166a1-01aa-4fcb-af21-c4acb6615894 None admin] RBAC: Authorizing `identity:get_user(user_id=d7228b454f0146fe92b200a4b05ec0cb)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:22.514474 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b166a1-01aa-4fcb-af21-c4acb6615894 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:22.516078 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 584/1151] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:14:22 2022] GET /identity/v3/users/d7228b454f0146fe92b200a4b05ec0cb => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:14:22.521906 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2bb143-efc1-4d02-8fea-b2b7a43eb2b1 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:22.563259 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:22.618849 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2bb143-efc1-4d02-8fea-b2b7a43eb2b1 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:22.650589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349c19a0-0a2b-43c2-8514-3c4c594855a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:22.651215 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c19a0-0a2b-43c2-8514-3c4c594855a4 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:22.651498 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c19a0-0a2b-43c2-8514-3c4c594855a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:22.651773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c19a0-0a2b-43c2-8514-3c4c594855a4 None admin] PATH_INFO: `/v3/projects/a7209d04d43448508d2beea46f10a2e9` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:22.652952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349c19a0-0a2b-43c2-8514-3c4c594855a4 None admin] RBAC: Authorizing `identity:get_project(project_id=a7209d04d43448508d2beea46f10a2e9)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:22.654860 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349c19a0-0a2b-43c2-8514-3c4c594855a4 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:22.656547 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 568/1152] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:14:22 2022] GET /identity/v3/projects/a7209d04d43448508d2beea46f10a2e9 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:14:22.662288 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b166a1-01aa-4fcb-af21-c4acb6615894 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:22.729872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:22.759657 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b166a1-01aa-4fcb-af21-c4acb6615894 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:22.793345 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db63f41-e406-476d-becf-30f44fe0766b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:22.793998 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db63f41-e406-476d-becf-30f44fe0766b None admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:22.794283 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db63f41-e406-476d-becf-30f44fe0766b None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:22.794558 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db63f41-e406-476d-becf-30f44fe0766b None admin] PATH_INFO: `/v3/projects/a7209d04d43448508d2beea46f10a2e9/users/d7228b454f0146fe92b200a4b05ec0cb/roles/7c88c815db854954b516ab97cd471647` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:22.796518 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db63f41-e406-476d-becf-30f44fe0766b None admin] RBAC: Authorizing `identity:create_grant(project_id=a7209d04d43448508d2beea46f10a2e9, user_id=d7228b454f0146fe92b200a4b05ec0cb, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:22.797940 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db63f41-e406-476d-becf-30f44fe0766b None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:22.889983 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 585/1153] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 15:14:22 2022] PUT /identity/v3/projects/a7209d04d43448508d2beea46f10a2e9/users/d7228b454f0146fe92b200a4b05ec0cb/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:14:24.830536 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10576587-6719-4379-8cf0-885c410922e9 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:24.831102 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10576587-6719-4379-8cf0-885c410922e9 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:24.831102 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10576587-6719-4379-8cf0-885c410922e9 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:24.832453 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 569/1154] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:14:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:14:24.837440 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb0c75-2da8-4262-920f-f3464596a9d1 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:24.837824 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb0c75-2da8-4262-920f-f3464596a9d1 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:24.837996 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb0c75-2da8-4262-920f-f3464596a9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:24.856155 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-7abb0c75-2da8-4262-920f-f3464596a9d1 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:14:24.857325 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-7abb0c75-2da8-4262-920f-f3464596a9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:14:24.920766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 586/1155] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:14:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:14:24.943655 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10576587-6719-4379-8cf0-885c410922e9 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:25.041815 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10576587-6719-4379-8cf0-885c410922e9 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:25.050784 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:25.074803 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f50633-6bfd-4a90-b05f-2a480ca4d155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:25.075428 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f50633-6bfd-4a90-b05f-2a480ca4d155 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:25.075712 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f50633-6bfd-4a90-b05f-2a480ca4d155 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:25.075990 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f50633-6bfd-4a90-b05f-2a480ca4d155 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:25.086240 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f50633-6bfd-4a90-b05f-2a480ca4d155 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:14:25.091466 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-a4f50633-6bfd-4a90-b05f-2a480ca4d155 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:14:25.091466 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:14:25.091466 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:14:25.091466 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:14:25.091466 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:14:25.091466 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:14:25.091466 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:14:25.091466 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:14:25.091466 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:14:25.091466 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:14:25.091466 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:14:25.091466 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:14:25.091466 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:14:25.091466 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:14:25.091466 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:14:25.091466 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:14:25.092589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:14:25.092589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:14:25.092589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:14:25.092589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:14:25.092589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:14:25.092589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:14:25.092589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:14:25.092589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:14:25.092589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:14:25.092589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:14:25.092589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:14:25.092589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:14:25.092589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:14:25.092589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:14:25.092589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:14:25.092589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:14:25.093624 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:14:25.093624 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:14:25.093624 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:14:25.093624 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:14:25.093624 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:14:25.093624 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:14:25.093624 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:14:25.093624 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:14:25.093624 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:14:25.093624 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:14:25.093624 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:14:25.093624 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:14:25.093624 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 570/1156] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 15:14:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:14:25.098748 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb0c75-2da8-4262-920f-f3464596a9d1 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:25.155799 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:25.196710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb0c75-2da8-4262-920f-f3464596a9d1 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:25.230168 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2469a2b0-5a66-439a-a77b-11d9ab5d677a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:25.230786 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469a2b0-5a66-439a-a77b-11d9ab5d677a None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:25.231576 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469a2b0-5a66-439a-a77b-11d9ab5d677a None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:25.231854 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469a2b0-5a66-439a-a77b-11d9ab5d677a None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:25.232597 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2469a2b0-5a66-439a-a77b-11d9ab5d677a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:14:25.232909 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2469a2b0-5a66-439a-a77b-11d9ab5d677a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:25.234337 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2469a2b0-5a66-439a-a77b-11d9ab5d677a None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:25.242319 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 587/1157] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:14:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:14:25.248912 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f50633-6bfd-4a90-b05f-2a480ca4d155 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:25.344762 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f50633-6bfd-4a90-b05f-2a480ca4d155 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:25.377757 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2711aa4-f7db-4dba-9283-eac36a86f8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:25.378380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2711aa4-f7db-4dba-9283-eac36a86f8ae None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:25.378664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2711aa4-f7db-4dba-9283-eac36a86f8ae None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:25.378943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2711aa4-f7db-4dba-9283-eac36a86f8ae None admin] PATH_INFO: `/v3/users/d7228b454f0146fe92b200a4b05ec0cb` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:25.380115 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2711aa4-f7db-4dba-9283-eac36a86f8ae None admin] RBAC: Authorizing `identity:get_user(user_id=d7228b454f0146fe92b200a4b05ec0cb)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:25.381539 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2711aa4-f7db-4dba-9283-eac36a86f8ae None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:25.383115 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 571/1158] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:14:25 2022] GET /identity/v3/users/d7228b454f0146fe92b200a4b05ec0cb => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:14:25.388676 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2469a2b0-5a66-439a-a77b-11d9ab5d677a None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:25.485171 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2469a2b0-5a66-439a-a77b-11d9ab5d677a None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:25.506481 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:25.517641 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede09964-4f50-40a4-a7df-34c202fc9618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:25.518569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede09964-4f50-40a4-a7df-34c202fc9618 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:25.518569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede09964-4f50-40a4-a7df-34c202fc9618 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:25.518782 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede09964-4f50-40a4-a7df-34c202fc9618 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:25.584438 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede09964-4f50-40a4-a7df-34c202fc9618 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:25.585917 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede09964-4f50-40a4-a7df-34c202fc9618 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:25.681413 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 588/1159] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:14:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 15:14:25.687129 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2711aa4-f7db-4dba-9283-eac36a86f8ae None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:25.721427 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:25.785503 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2711aa4-f7db-4dba-9283-eac36a86f8ae None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:25.820518 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c3301-604b-4fa8-8f1e-650d961735fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:25.821149 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c3301-604b-4fa8-8f1e-650d961735fc None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:25.821431 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c3301-604b-4fa8-8f1e-650d961735fc None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:25.821707 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c3301-604b-4fa8-8f1e-650d961735fc None admin] PATH_INFO: `/v3/projects/a7209d04d43448508d2beea46f10a2e9` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:25.822849 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13c3301-604b-4fa8-8f1e-650d961735fc None admin] RBAC: Authorizing `identity:get_project(project_id=a7209d04d43448508d2beea46f10a2e9)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:25.824285 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13c3301-604b-4fa8-8f1e-650d961735fc None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:25.826540 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 572/1160] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:14:25 2022] GET /identity/v3/projects/a7209d04d43448508d2beea46f10a2e9 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:14:25.832664 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede09964-4f50-40a4-a7df-34c202fc9618 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:25.856120 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:25.928968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede09964-4f50-40a4-a7df-34c202fc9618 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:25.961501 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf5dcde-746e-4456-afed-792cd90af65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:25.962039 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf5dcde-746e-4456-afed-792cd90af65b None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:25.962319 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf5dcde-746e-4456-afed-792cd90af65b None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:25.962594 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf5dcde-746e-4456-afed-792cd90af65b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:25.963364 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf5dcde-746e-4456-afed-792cd90af65b None admin] RBAC: Adding query filter params (role.id=7c88c815db854954b516ab97cd471647, scope.project.id=a7209d04d43448508d2beea46f10a2e9, user.id=d7228b454f0146fe92b200a4b05ec0cb) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:14:25.963674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf5dcde-746e-4456-afed-792cd90af65b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:25.965126 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf5dcde-746e-4456-afed-792cd90af65b None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:25.971268 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 589/1161] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:14:25 2022] GET /identity/v3/role_assignments?user.id=d7228b454f0146fe92b200a4b05ec0cb&scope.project.id=a7209d04d43448508d2beea46f10a2e9&role.id=7c88c815db854954b516ab97cd471647 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:14:27.962426 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34059e01-3b08-4edb-b3bd-780cecb7897e None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:27.962951 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34059e01-3b08-4edb-b3bd-780cecb7897e None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:27.963063 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34059e01-3b08-4edb-b3bd-780cecb7897e None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:27.964404 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 573/1162] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:14:27.969539 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d18ce-9f11-4897-a08b-3b857f486b8e None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:27.969961 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d18ce-9f11-4897-a08b-3b857f486b8e None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:27.970095 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d18ce-9f11-4897-a08b-3b857f486b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:27.987631 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-d62d18ce-9f11-4897-a08b-3b857f486b8e None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:14:27.988841 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-d62d18ce-9f11-4897-a08b-3b857f486b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:14:28.054243 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 590/1163] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:14:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:14:28.076981 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34059e01-3b08-4edb-b3bd-780cecb7897e None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:28.186074 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34059e01-3b08-4edb-b3bd-780cecb7897e None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:28.188400 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:28.218037 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2641ed5-06bd-4a6d-bd80-2addf7f6f7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:28.218588 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2641ed5-06bd-4a6d-bd80-2addf7f6f7fe None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:28.218873 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2641ed5-06bd-4a6d-bd80-2addf7f6f7fe None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:28.219154 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2641ed5-06bd-4a6d-bd80-2addf7f6f7fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:28.285498 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2641ed5-06bd-4a6d-bd80-2addf7f6f7fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:28.286924 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2641ed5-06bd-4a6d-bd80-2addf7f6f7fe None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:28.384824 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 574/1164] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:14:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 15:14:28.391015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d18ce-9f11-4897-a08b-3b857f486b8e None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:28.408439 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:28.490367 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d18ce-9f11-4897-a08b-3b857f486b8e None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:28.528884 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f10f8-8a19-4bed-ab17-cbf11397d438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:28.529521 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f10f8-8a19-4bed-ab17-cbf11397d438 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:28.529803 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f10f8-8a19-4bed-ab17-cbf11397d438 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:28.530076 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f10f8-8a19-4bed-ab17-cbf11397d438 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:28.536651 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f10f8-8a19-4bed-ab17-cbf11397d438 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:28.538106 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f10f8-8a19-4bed-ab17-cbf11397d438 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:28.550001 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-e83f10f8-8a19-4bed-ab17-cbf11397d438 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 05 15:14:28.550001 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:14:28.550001 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 05 15:14:28.550001 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 05 15:14:28.550001 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 15:14:28.550001 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 15:14:28.550001 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 15:14:28.550001 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 15:14:28.550001 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 05 15:14:28.550001 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:14:28.550001 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 15:14:28.550001 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:14:28.550001 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:14:28.550001 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:14:28.550001 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:14:28.550001 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:14:28.550001 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:14:28.551223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:14:28.551223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:14:28.551223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:14:28.551223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:14:28.551223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:14:28.551223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:14:28.551223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:14:28.551223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:14:28.551223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:14:28.551223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:14:28.551223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:14:28.551223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:14:28.551223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:14:28.551223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:14:28.551223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:14:28.551223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:14:28.552297 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:14:28.552297 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:14:28.552297 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:14:28.552297 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:14:28.552297 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:14:28.552297 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:14:28.552297 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:14:28.552297 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:14:28.552297 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:14:28.552297 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:14:28.552297 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 05 15:14:28.552297 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:14:28.552297 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 05 15:14:28.552297 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:14:28.552297 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 591/1165] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Aug 5 15:14:28 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:14:28.557131 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2641ed5-06bd-4a6d-bd80-2addf7f6f7fe None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:28.560217 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:28.651925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2641ed5-06bd-4a6d-bd80-2addf7f6f7fe None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:28.684817 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475c0c6-e081-4ae7-8d19-2fa664106457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:28.685408 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475c0c6-e081-4ae7-8d19-2fa664106457 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:28.685688 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475c0c6-e081-4ae7-8d19-2fa664106457 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:28.685961 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475c0c6-e081-4ae7-8d19-2fa664106457 None admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:28.686642 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7475c0c6-e081-4ae7-8d19-2fa664106457 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:14:28.686944 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7475c0c6-e081-4ae7-8d19-2fa664106457 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:28.688308 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7475c0c6-e081-4ae7-8d19-2fa664106457 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:28.695092 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 575/1166] 10.222.0.40 () {56 vars in 1115 bytes} [Fri Aug 5 15:14:28 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:14:30.657968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2e1cc-d1b1-4b09-9d28-50deee34c700 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:30.658536 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2e1cc-d1b1-4b09-9d28-50deee34c700 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:30.658536 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2e1cc-d1b1-4b09-9d28-50deee34c700 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:30.659868 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 592/1167] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:14:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:14:30.665174 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377dcc1e-6735-419f-9754-07ae6d5b0fa4 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:30.665607 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377dcc1e-6735-419f-9754-07ae6d5b0fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:30.665741 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377dcc1e-6735-419f-9754-07ae6d5b0fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:30.683355 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-377dcc1e-6735-419f-9754-07ae6d5b0fa4 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:14:30.685081 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-377dcc1e-6735-419f-9754-07ae6d5b0fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:14:30.749536 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:30.755145 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 576/1168] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:14:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:14:30.778398 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad2e1cc-d1b1-4b09-9d28-50deee34c700 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:30.853398 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:30.876691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad2e1cc-d1b1-4b09-9d28-50deee34c700 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:30.909252 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08308650-193d-4776-82a7-61d33e35d478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:30.909803 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08308650-193d-4776-82a7-61d33e35d478 None admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:30.910089 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08308650-193d-4776-82a7-61d33e35d478 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:30.910370 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08308650-193d-4776-82a7-61d33e35d478 None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:30.911075 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08308650-193d-4776-82a7-61d33e35d478 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:30.912504 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08308650-193d-4776-82a7-61d33e35d478 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:30.964540 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 593/1169] 10.222.0.40 () {60 vars in 1138 bytes} [Fri Aug 5 15:14:30 2022] POST /identity/v3/domains => generated 267 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:14:32.959497 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209a054-b7d1-45e3-9df0-6ea78101ad15 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:32.960033 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209a054-b7d1-45e3-9df0-6ea78101ad15 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:32.960033 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209a054-b7d1-45e3-9df0-6ea78101ad15 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:32.961438 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 577/1170] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:14:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:14:32.967182 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db374f13-bbb4-43de-8329-fd8b5fa768ec None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:32.967571 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db374f13-bbb4-43de-8329-fd8b5fa768ec None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:32.967744 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db374f13-bbb4-43de-8329-fd8b5fa768ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:32.985910 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-db374f13-bbb4-43de-8329-fd8b5fa768ec None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:14:32.987081 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-db374f13-bbb4-43de-8329-fd8b5fa768ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:14:33.050989 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 594/1171] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:14:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:14:33.077717 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6209a054-b7d1-45e3-9df0-6ea78101ad15 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:33.179018 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6209a054-b7d1-45e3-9df0-6ea78101ad15 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:33.211525 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed3cb4e-18e3-4c97-81b3-78b972d056f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:33.212111 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3cb4e-18e3-4c97-81b3-78b972d056f1 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:33.212398 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3cb4e-18e3-4c97-81b3-78b972d056f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:33.212684 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3cb4e-18e3-4c97-81b3-78b972d056f1 None admin] PATH_INFO: `/v3/domains/1a4cbe05b8ab4f95850de901f02167d6` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:33.223323 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed3cb4e-18e3-4c97-81b3-78b972d056f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=1a4cbe05b8ab4f95850de901f02167d6)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:33.224800 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed3cb4e-18e3-4c97-81b3-78b972d056f1 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:33.226421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 578/1172] 10.222.0.40 () {56 vars in 1183 bytes} [Fri Aug 5 15:14:33 2022] GET /identity/v3/domains/1a4cbe05b8ab4f95850de901f02167d6 => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:14:33.233049 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db374f13-bbb4-43de-8329-fd8b5fa768ec None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:33.295941 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:33.329225 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db374f13-bbb4-43de-8329-fd8b5fa768ec None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:33.362514 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9f99e7-73d3-4a9a-942e-e85f50b63185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:33.363130 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f99e7-73d3-4a9a-942e-e85f50b63185 None admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:33.363416 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f99e7-73d3-4a9a-942e-e85f50b63185 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:33.363697 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f99e7-73d3-4a9a-942e-e85f50b63185 None admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:33.364412 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9f99e7-73d3-4a9a-942e-e85f50b63185 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:33.365839 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9f99e7-73d3-4a9a-942e-e85f50b63185 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:33.491183 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 595/1173] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 5 15:14:33 2022] POST /identity/v3/projects => generated 361 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:14:35.515911 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b3195-493e-4146-97a1-9e7ca16b1c54 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:35.516473 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b3195-493e-4146-97a1-9e7ca16b1c54 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:35.516570 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b3195-493e-4146-97a1-9e7ca16b1c54 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:35.517886 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 579/1174] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:14:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:14:35.523022 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a4f07-265d-4a1a-8ba2-1554e779c7e0 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:35.523456 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a4f07-265d-4a1a-8ba2-1554e779c7e0 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:35.523592 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a4f07-265d-4a1a-8ba2-1554e779c7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:35.541520 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-023a4f07-265d-4a1a-8ba2-1554e779c7e0 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:14:35.543242 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-023a4f07-265d-4a1a-8ba2-1554e779c7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:14:35.608247 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 596/1175] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:14:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:14:35.632946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b3195-493e-4146-97a1-9e7ca16b1c54 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:35.691318 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:35.732118 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b3195-493e-4146-97a1-9e7ca16b1c54 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:35.764671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3aa36-bd2b-443e-8b91-a3a9165865ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:35.765291 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3aa36-bd2b-443e-8b91-a3a9165865ce None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:35.765575 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3aa36-bd2b-443e-8b91-a3a9165865ce None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:35.765853 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3aa36-bd2b-443e-8b91-a3a9165865ce None admin] PATH_INFO: `/v3/domains/1a4cbe05b8ab4f95850de901f02167d6` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:35.767048 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c3aa36-bd2b-443e-8b91-a3a9165865ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=1a4cbe05b8ab4f95850de901f02167d6)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:35.768443 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c3aa36-bd2b-443e-8b91-a3a9165865ce None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:35.770010 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 580/1176] 10.222.0.40 () {56 vars in 1183 bytes} [Fri Aug 5 15:14:35 2022] GET /identity/v3/domains/1a4cbe05b8ab4f95850de901f02167d6 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:14:35.775666 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a4f07-265d-4a1a-8ba2-1554e779c7e0 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:35.824317 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:35.874742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a4f07-265d-4a1a-8ba2-1554e779c7e0 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:35.907562 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bda45d9-1fd0-4789-b502-daa5e87293bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:35.908209 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda45d9-1fd0-4789-b502-daa5e87293bd None admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:35.908497 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda45d9-1fd0-4789-b502-daa5e87293bd None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:35.908773 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda45d9-1fd0-4789-b502-daa5e87293bd None admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:35.909454 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bda45d9-1fd0-4789-b502-daa5e87293bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:35.910890 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bda45d9-1fd0-4789-b502-daa5e87293bd None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:35.986684 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 597/1177] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 5 15:14:35 2022] POST /identity/v3/users => generated 311 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:14:38.020021 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb7487-f266-40ae-9d2a-9f6f2ae155c7 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:38.020570 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb7487-f266-40ae-9d2a-9f6f2ae155c7 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:38.020684 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb7487-f266-40ae-9d2a-9f6f2ae155c7 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:38.021964 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 581/1178] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:14:38.027652 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea5a53-8288-4332-8dad-43c01698a363 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:38.028093 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea5a53-8288-4332-8dad-43c01698a363 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:38.028231 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea5a53-8288-4332-8dad-43c01698a363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:38.045757 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-34ea5a53-8288-4332-8dad-43c01698a363 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:14:38.046940 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-34ea5a53-8288-4332-8dad-43c01698a363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:14:38.110908 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 598/1179] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:14:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:14:38.134428 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fb7487-f266-40ae-9d2a-9f6f2ae155c7 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:38.236059 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fb7487-f266-40ae-9d2a-9f6f2ae155c7 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:38.260868 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:38.269744 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80feec8a-9ed1-408e-a0ed-87fe9e325fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:38.270421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80feec8a-9ed1-408e-a0ed-87fe9e325fda None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:38.270663 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80feec8a-9ed1-408e-a0ed-87fe9e325fda None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:38.270944 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80feec8a-9ed1-408e-a0ed-87fe9e325fda None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:38.281237 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80feec8a-9ed1-408e-a0ed-87fe9e325fda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:14:38.286089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-80feec8a-9ed1-408e-a0ed-87fe9e325fda None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:14:38.286089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:14:38.286089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:14:38.286089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:14:38.286089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:14:38.286089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:14:38.286089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:14:38.286089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:14:38.286089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:14:38.286089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:14:38.286089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:14:38.286089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:14:38.286089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:14:38.286089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:14:38.286089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:14:38.286089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:14:38.287180 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:14:38.287180 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:14:38.287180 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:14:38.287180 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:14:38.287180 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:14:38.287180 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:14:38.287180 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:14:38.287180 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:14:38.287180 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:14:38.287180 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:14:38.287180 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:14:38.287180 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:14:38.287180 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:14:38.287180 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:14:38.287180 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:14:38.287180 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:14:38.288270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:14:38.288270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:14:38.288270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:14:38.288270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:14:38.288270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:14:38.288270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:14:38.288270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:14:38.288270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:14:38.288270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:14:38.288270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:14:38.288270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:14:38.288270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:14:38.288270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 582/1180] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 15:14:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:14:38.293552 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea5a53-8288-4332-8dad-43c01698a363 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:38.326314 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:38.389946 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea5a53-8288-4332-8dad-43c01698a363 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:38.424639 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d19963-5780-4e9c-80bd-172c95fce5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:38.425515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d19963-5780-4e9c-80bd-172c95fce5a9 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:38.425865 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d19963-5780-4e9c-80bd-172c95fce5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:38.426173 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d19963-5780-4e9c-80bd-172c95fce5a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:38.426899 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d19963-5780-4e9c-80bd-172c95fce5a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:14:38.427213 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d19963-5780-4e9c-80bd-172c95fce5a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:38.428943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d19963-5780-4e9c-80bd-172c95fce5a9 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:38.438218 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 599/1181] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:14:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:14:38.444009 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80feec8a-9ed1-408e-a0ed-87fe9e325fda None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:38.547944 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80feec8a-9ed1-408e-a0ed-87fe9e325fda None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:38.580478 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb51eb9-846a-4f38-8107-b78fa6ecd072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:38.581110 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb51eb9-846a-4f38-8107-b78fa6ecd072 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:38.581398 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb51eb9-846a-4f38-8107-b78fa6ecd072 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:38.581682 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb51eb9-846a-4f38-8107-b78fa6ecd072 None admin] PATH_INFO: `/v3/users/4b4b9af39ea944bea7ee245c62b862e1` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:38.595088 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb51eb9-846a-4f38-8107-b78fa6ecd072 None admin] RBAC: Authorizing `identity:get_user(user_id=4b4b9af39ea944bea7ee245c62b862e1)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:38.596550 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb51eb9-846a-4f38-8107-b78fa6ecd072 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:38.598170 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 583/1182] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:14:38 2022] GET /identity/v3/users/4b4b9af39ea944bea7ee245c62b862e1 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:14:38.603981 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d19963-5780-4e9c-80bd-172c95fce5a9 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:38.696146 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:38.701858 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d19963-5780-4e9c-80bd-172c95fce5a9 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:38.734089 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43bb2b-7308-4178-9a33-b029df359ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:38.734696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43bb2b-7308-4178-9a33-b029df359ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:38.734979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43bb2b-7308-4178-9a33-b029df359ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:38.735255 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43bb2b-7308-4178-9a33-b029df359ae4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:38.800341 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b43bb2b-7308-4178-9a33-b029df359ae4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:38.801737 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b43bb2b-7308-4178-9a33-b029df359ae4 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:38.890636 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:38.896937 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 600/1183] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:14:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 15:14:38.903260 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb51eb9-846a-4f38-8107-b78fa6ecd072 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:38.947699 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:39.000736 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb51eb9-846a-4f38-8107-b78fa6ecd072 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:39.033545 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b4055-7ae1-4b7a-83ab-8ff0a134e1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:39.034176 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b4055-7ae1-4b7a-83ab-8ff0a134e1e7 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:39.034465 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b4055-7ae1-4b7a-83ab-8ff0a134e1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:39.034743 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b4055-7ae1-4b7a-83ab-8ff0a134e1e7 None admin] PATH_INFO: `/v3/projects/10712085d1a3451095b35610eba7f5f5` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:39.035901 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146b4055-7ae1-4b7a-83ab-8ff0a134e1e7 None admin] RBAC: Authorizing `identity:get_project(project_id=10712085d1a3451095b35610eba7f5f5)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:39.037338 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146b4055-7ae1-4b7a-83ab-8ff0a134e1e7 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:39.038973 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 584/1184] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:14:38 2022] GET /identity/v3/projects/10712085d1a3451095b35610eba7f5f5 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:14:39.044959 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43bb2b-7308-4178-9a33-b029df359ae4 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:39.141345 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43bb2b-7308-4178-9a33-b029df359ae4 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:39.175865 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e536e-928b-49fe-b20c-1203773e926c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:39.176511 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e536e-928b-49fe-b20c-1203773e926c None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:39.176793 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e536e-928b-49fe-b20c-1203773e926c None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:39.177063 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e536e-928b-49fe-b20c-1203773e926c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:39.177843 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3e536e-928b-49fe-b20c-1203773e926c None admin] RBAC: Adding query filter params (role.id=7c88c815db854954b516ab97cd471647, scope.project.id=10712085d1a3451095b35610eba7f5f5, user.id=4b4b9af39ea944bea7ee245c62b862e1) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:14:39.178153 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3e536e-928b-49fe-b20c-1203773e926c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:39.179570 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3e536e-928b-49fe-b20c-1203773e926c None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:39.185398 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 601/1185] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:14:39 2022] GET /identity/v3/role_assignments?user.id=4b4b9af39ea944bea7ee245c62b862e1&scope.project.id=10712085d1a3451095b35610eba7f5f5&role.id=7c88c815db854954b516ab97cd471647 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:14:41.160909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65875de-2b8e-49fa-9f72-845d4be5b0fc None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:41.161447 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65875de-2b8e-49fa-9f72-845d4be5b0fc None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:41.161447 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65875de-2b8e-49fa-9f72-845d4be5b0fc None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:41.162813 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 585/1186] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:14:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:14:41.168645 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c362ad-6ced-48d7-bc00-751d8be5b5ca None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:41.169076 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c362ad-6ced-48d7-bc00-751d8be5b5ca None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:41.169660 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c362ad-6ced-48d7-bc00-751d8be5b5ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:41.186784 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-95c362ad-6ced-48d7-bc00-751d8be5b5ca None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:14:41.187962 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-95c362ad-6ced-48d7-bc00-751d8be5b5ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:14:41.241785 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:41.252930 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 602/1187] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:14:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:14:41.274680 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65875de-2b8e-49fa-9f72-845d4be5b0fc None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:41.373381 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65875de-2b8e-49fa-9f72-845d4be5b0fc None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:41.385786 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:41.406410 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d0d22-cef7-48e6-a3f6-b6bef3202e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:41.407046 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d0d22-cef7-48e6-a3f6-b6bef3202e4c None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:41.407332 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d0d22-cef7-48e6-a3f6-b6bef3202e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:41.407658 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d0d22-cef7-48e6-a3f6-b6bef3202e4c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:41.419535 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d0d22-cef7-48e6-a3f6-b6bef3202e4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:14:41.424438 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-640d0d22-cef7-48e6-a3f6-b6bef3202e4c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:14:41.424438 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:14:41.424438 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:14:41.424438 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:14:41.424438 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:14:41.424438 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:14:41.424438 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:14:41.424438 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:14:41.424438 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:14:41.424438 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:14:41.424438 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:14:41.424438 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:14:41.424438 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:14:41.424438 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:14:41.424438 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:14:41.424438 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:14:41.425517 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:14:41.425517 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:14:41.425517 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:14:41.425517 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:14:41.425517 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:14:41.425517 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:14:41.425517 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:14:41.425517 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:14:41.425517 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:14:41.425517 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:14:41.425517 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:14:41.425517 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:14:41.425517 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:14:41.425517 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:14:41.425517 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:14:41.425517 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:14:41.426381 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:14:41.426381 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:14:41.426381 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:14:41.426381 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:14:41.426381 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:14:41.426381 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:14:41.426381 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:14:41.426381 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:14:41.426381 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:14:41.426381 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:14:41.426381 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:14:41.426381 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:14:41.426381 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 586/1188] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 15:14:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:14:41.432820 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c362ad-6ced-48d7-bc00-751d8be5b5ca None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:41.527811 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c362ad-6ced-48d7-bc00-751d8be5b5ca None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:41.559755 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd1cd28-1967-416c-9278-6a6373a6d221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:41.560545 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd1cd28-1967-416c-9278-6a6373a6d221 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:41.560825 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd1cd28-1967-416c-9278-6a6373a6d221 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:41.561101 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd1cd28-1967-416c-9278-6a6373a6d221 None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:41.561837 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd1cd28-1967-416c-9278-6a6373a6d221 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:14:41.562142 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd1cd28-1967-416c-9278-6a6373a6d221 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:41.563497 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd1cd28-1967-416c-9278-6a6373a6d221 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:41.571214 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 603/1189] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:14:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:14:41.577432 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d0d22-cef7-48e6-a3f6-b6bef3202e4c None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:41.678071 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d0d22-cef7-48e6-a3f6-b6bef3202e4c None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:41.712874 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e666a0-9694-4f93-af64-0d794078ec8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:41.713492 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e666a0-9694-4f93-af64-0d794078ec8e None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:41.714356 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e666a0-9694-4f93-af64-0d794078ec8e None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:41.714649 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e666a0-9694-4f93-af64-0d794078ec8e None admin] PATH_INFO: `/v3/users/4b4b9af39ea944bea7ee245c62b862e1` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:41.715798 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e666a0-9694-4f93-af64-0d794078ec8e None admin] RBAC: Authorizing `identity:get_user(user_id=4b4b9af39ea944bea7ee245c62b862e1)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:41.717258 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e666a0-9694-4f93-af64-0d794078ec8e None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:41.718837 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 587/1190] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:14:41 2022] GET /identity/v3/users/4b4b9af39ea944bea7ee245c62b862e1 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:14:41.724583 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd1cd28-1967-416c-9278-6a6373a6d221 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:41.775931 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:41.821190 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd1cd28-1967-416c-9278-6a6373a6d221 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:41.854258 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b3e2a1-2e16-4ebb-95ad-8f1917673f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:41.854912 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b3e2a1-2e16-4ebb-95ad-8f1917673f41 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:41.855198 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b3e2a1-2e16-4ebb-95ad-8f1917673f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:41.855479 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b3e2a1-2e16-4ebb-95ad-8f1917673f41 None admin] PATH_INFO: `/v3/projects/10712085d1a3451095b35610eba7f5f5` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:41.856657 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b3e2a1-2e16-4ebb-95ad-8f1917673f41 None admin] RBAC: Authorizing `identity:get_project(project_id=10712085d1a3451095b35610eba7f5f5)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:41.858083 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b3e2a1-2e16-4ebb-95ad-8f1917673f41 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:41.859723 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 604/1191] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:14:41 2022] GET /identity/v3/projects/10712085d1a3451095b35610eba7f5f5 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:14:41.865489 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e666a0-9694-4f93-af64-0d794078ec8e None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:41.907278 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:41.962011 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e666a0-9694-4f93-af64-0d794078ec8e None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:41.993647 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22a511-8287-4c51-947d-c7e9e2cf1272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:41.994284 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22a511-8287-4c51-947d-c7e9e2cf1272 None admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:41.994907 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22a511-8287-4c51-947d-c7e9e2cf1272 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:41.995709 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22a511-8287-4c51-947d-c7e9e2cf1272 None admin] PATH_INFO: `/v3/projects/10712085d1a3451095b35610eba7f5f5/users/4b4b9af39ea944bea7ee245c62b862e1/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:41.997655 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f22a511-8287-4c51-947d-c7e9e2cf1272 None admin] RBAC: Authorizing `identity:create_grant(project_id=10712085d1a3451095b35610eba7f5f5, user_id=4b4b9af39ea944bea7ee245c62b862e1, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:41.999060 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f22a511-8287-4c51-947d-c7e9e2cf1272 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:42.065214 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 588/1192] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 15:14:41 2022] PUT /identity/v3/projects/10712085d1a3451095b35610eba7f5f5/users/4b4b9af39ea944bea7ee245c62b862e1/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:14:44.023857 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d596e-29ce-4a6f-bcf5-3f38e9d70161 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:44.024442 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d596e-29ce-4a6f-bcf5-3f38e9d70161 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:44.024547 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d596e-29ce-4a6f-bcf5-3f38e9d70161 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:44.025832 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 605/1193] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:14:44.031690 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151780ad-c247-4c84-83f0-77d01c95e28c None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:44.032108 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151780ad-c247-4c84-83f0-77d01c95e28c None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:44.032272 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151780ad-c247-4c84-83f0-77d01c95e28c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:44.049732 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-151780ad-c247-4c84-83f0-77d01c95e28c None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:14:44.050902 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-151780ad-c247-4c84-83f0-77d01c95e28c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:14:44.121322 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 589/1194] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:14:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:14:44.145694 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d596e-29ce-4a6f-bcf5-3f38e9d70161 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:44.244643 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d596e-29ce-4a6f-bcf5-3f38e9d70161 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:44.264514 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:44.278307 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb3507-5f4e-46b9-93d4-4034702005cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:44.278503 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb3507-5f4e-46b9-93d4-4034702005cf None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:44.278630 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb3507-5f4e-46b9-93d4-4034702005cf None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:44.278867 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb3507-5f4e-46b9-93d4-4034702005cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:44.288955 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeb3507-5f4e-46b9-93d4-4034702005cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:14:44.293857 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-4aeb3507-5f4e-46b9-93d4-4034702005cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:14:44.293857 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:14:44.293857 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:14:44.293857 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:14:44.293857 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:14:44.293857 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:14:44.293857 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:14:44.293857 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:14:44.293857 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:14:44.293857 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:14:44.293857 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:14:44.293857 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:14:44.293857 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:14:44.293857 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:14:44.293857 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:14:44.293857 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:14:44.295002 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:14:44.295002 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:14:44.295002 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:14:44.295002 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:14:44.295002 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:14:44.295002 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:14:44.295002 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:14:44.295002 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:14:44.295002 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:14:44.295002 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:14:44.295002 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:14:44.295002 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:14:44.295002 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:14:44.295002 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:14:44.295002 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:14:44.295002 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:14:44.296131 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:14:44.296131 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:14:44.296131 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:14:44.296131 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:14:44.296131 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:14:44.296131 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:14:44.296131 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:14:44.296131 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:14:44.296131 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:14:44.296131 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:14:44.296131 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:14:44.296131 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:14:44.296131 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 606/1195] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 15:14:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:14:44.301211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151780ad-c247-4c84-83f0-77d01c95e28c None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:44.330272 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:44.397994 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151780ad-c247-4c84-83f0-77d01c95e28c None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:44.430344 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a888b88-7b0b-49c3-acd2-b3bb02e6a57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:44.430950 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a888b88-7b0b-49c3-acd2-b3bb02e6a57b None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:44.431235 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a888b88-7b0b-49c3-acd2-b3bb02e6a57b None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:44.431511 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a888b88-7b0b-49c3-acd2-b3bb02e6a57b None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:44.432247 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a888b88-7b0b-49c3-acd2-b3bb02e6a57b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:14:44.433066 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a888b88-7b0b-49c3-acd2-b3bb02e6a57b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:44.434424 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a888b88-7b0b-49c3-acd2-b3bb02e6a57b None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:44.442222 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 590/1196] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:14:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:14:44.448695 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb3507-5f4e-46b9-93d4-4034702005cf None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:44.546080 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb3507-5f4e-46b9-93d4-4034702005cf None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:44.579459 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ce648-ca09-4205-b48d-5063b1f4af2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:44.580113 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ce648-ca09-4205-b48d-5063b1f4af2d None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:44.580401 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ce648-ca09-4205-b48d-5063b1f4af2d None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:44.580676 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ce648-ca09-4205-b48d-5063b1f4af2d None admin] PATH_INFO: `/v3/users/4b4b9af39ea944bea7ee245c62b862e1` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:44.581822 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976ce648-ca09-4205-b48d-5063b1f4af2d None admin] RBAC: Authorizing `identity:get_user(user_id=4b4b9af39ea944bea7ee245c62b862e1)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:44.583296 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976ce648-ca09-4205-b48d-5063b1f4af2d None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:44.584946 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 607/1197] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 5 15:14:44 2022] GET /identity/v3/users/4b4b9af39ea944bea7ee245c62b862e1 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:14:44.590909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a888b88-7b0b-49c3-acd2-b3bb02e6a57b None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:44.678796 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:44.687603 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a888b88-7b0b-49c3-acd2-b3bb02e6a57b None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:44.719652 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47ca592-b542-48e0-9ea2-c83ed30c6a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:44.720276 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47ca592-b542-48e0-9ea2-c83ed30c6a71 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:44.720559 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47ca592-b542-48e0-9ea2-c83ed30c6a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:44.720832 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47ca592-b542-48e0-9ea2-c83ed30c6a71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:44.784652 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47ca592-b542-48e0-9ea2-c83ed30c6a71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:44.786078 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47ca592-b542-48e0-9ea2-c83ed30c6a71 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:44.871590 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:44.880533 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 591/1198] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:14:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 15:14:44.886297 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ce648-ca09-4205-b48d-5063b1f4af2d None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:44.934528 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:44.986161 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ce648-ca09-4205-b48d-5063b1f4af2d None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:45.017882 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be5b318-356d-4c55-9681-cac011c3fbd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:45.018501 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be5b318-356d-4c55-9681-cac011c3fbd7 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:45.018781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be5b318-356d-4c55-9681-cac011c3fbd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:45.019055 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be5b318-356d-4c55-9681-cac011c3fbd7 None admin] PATH_INFO: `/v3/projects/10712085d1a3451095b35610eba7f5f5` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:45.020210 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be5b318-356d-4c55-9681-cac011c3fbd7 None admin] RBAC: Authorizing `identity:get_project(project_id=10712085d1a3451095b35610eba7f5f5)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:45.021645 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be5b318-356d-4c55-9681-cac011c3fbd7 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:45.023278 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 608/1199] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 5 15:14:44 2022] GET /identity/v3/projects/10712085d1a3451095b35610eba7f5f5 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:14:45.029771 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47ca592-b542-48e0-9ea2-c83ed30c6a71 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:14:45.125102 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47ca592-b542-48e0-9ea2-c83ed30c6a71 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:14:45.158333 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0023e39-42c5-47ec-8fe0-59704370b17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:14:45.158933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0023e39-42c5-47ec-8fe0-59704370b17e None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:14:45.159217 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0023e39-42c5-47ec-8fe0-59704370b17e None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:14:45.159509 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0023e39-42c5-47ec-8fe0-59704370b17e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:14:45.160300 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0023e39-42c5-47ec-8fe0-59704370b17e None admin] RBAC: Adding query filter params (role.id=7c88c815db854954b516ab97cd471647, scope.project.id=10712085d1a3451095b35610eba7f5f5, user.id=4b4b9af39ea944bea7ee245c62b862e1) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:14:45.160616 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0023e39-42c5-47ec-8fe0-59704370b17e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:14:45.162038 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0023e39-42c5-47ec-8fe0-59704370b17e None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:14:45.168407 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 592/1200] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:14:45 2022] GET /identity/v3/role_assignments?user.id=4b4b9af39ea944bea7ee245c62b862e1&scope.project.id=10712085d1a3451095b35610eba7f5f5&role.id=7c88c815db854954b516ab97cd471647 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:14:55.013858 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=42 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:14:55.165146 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=72 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:15:56.061805 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4a789-9583-4ba3-9ad3-135739a75468 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:15:56.062395 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4a789-9583-4ba3-9ad3-135739a75468 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:15:56.062395 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4a789-9583-4ba3-9ad3-135739a75468 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:15:56.063699 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 609/1201] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:15:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:15:56.069007 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbaad7d-63a6-4486-a370-eddf74ef1554 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:15:56.069474 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbaad7d-63a6-4486-a370-eddf74ef1554 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:15:56.069677 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbaad7d-63a6-4486-a370-eddf74ef1554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:15:56.088474 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-6bbaad7d-63a6-4486-a370-eddf74ef1554 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:15:56.089738 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-6bbaad7d-63a6-4486-a370-eddf74ef1554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:15:56.168916 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 593/1202] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:15:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 15:15:56.192943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4a789-9583-4ba3-9ad3-135739a75468 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:15:56.300448 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4a789-9583-4ba3-9ad3-135739a75468 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:15:56.335358 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb2b68-7027-4698-b524-3819a9cb1856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:15:56.335966 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb2b68-7027-4698-b524-3819a9cb1856 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:15:56.336279 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb2b68-7027-4698-b524-3819a9cb1856 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:15:56.336555 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb2b68-7027-4698-b524-3819a9cb1856 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:15:56.347640 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abb2b68-7027-4698-b524-3819a9cb1856 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:15:56.349104 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abb2b68-7027-4698-b524-3819a9cb1856 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:15:56.363729 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-0abb2b68-7027-4698-b524-3819a9cb1856 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:15:56.363729 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:15:56.363729 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:15:56.363729 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:15:56.363729 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:15:56.363729 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:15:56.363729 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:15:56.363729 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:15:56.363729 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:15:56.363729 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:15:56.363729 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:15:56.363729 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:15:56.363729 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:15:56.363729 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:15:56.363729 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:15:56.364904 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:15:56.364904 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:15:56.364904 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:15:56.364904 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:15:56.364904 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:15:56.364904 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:15:56.364904 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:15:56.364904 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:15:56.364904 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:15:56.364904 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:15:56.364904 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:15:56.364904 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:15:56.364904 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:15:56.364904 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:15:56.364904 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:15:56.364904 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:15:56.364904 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:15:56.365983 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:15:56.365983 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:15:56.365983 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:15:56.365983 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:15:56.365983 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 610/1203] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:15:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:15:56.371438 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbaad7d-63a6-4486-a370-eddf74ef1554 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:15:56.473100 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbaad7d-63a6-4486-a370-eddf74ef1554 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:15:56.501121 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:15:56.506782 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37525b-438f-4cce-bcdb-3d16f54adfe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:15:56.507391 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37525b-438f-4cce-bcdb-3d16f54adfe5 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:15:56.507672 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37525b-438f-4cce-bcdb-3d16f54adfe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:15:56.507944 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37525b-438f-4cce-bcdb-3d16f54adfe5 None admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:15:56.508653 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc37525b-438f-4cce-bcdb-3d16f54adfe5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:15:56.508955 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc37525b-438f-4cce-bcdb-3d16f54adfe5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:15:56.510304 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc37525b-438f-4cce-bcdb-3d16f54adfe5 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:15:56.521041 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 594/1204] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:15:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:15:56.527005 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb2b68-7027-4698-b524-3819a9cb1856 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:15:56.588612 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:15:56.639803 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb2b68-7027-4698-b524-3819a9cb1856 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:15:56.679047 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aabb00e-95a7-4698-a102-1ff22315a3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:15:56.679651 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aabb00e-95a7-4698-a102-1ff22315a3f8 None admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:15:56.679932 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aabb00e-95a7-4698-a102-1ff22315a3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:15:56.680240 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aabb00e-95a7-4698-a102-1ff22315a3f8 None admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:15:56.680925 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aabb00e-95a7-4698-a102-1ff22315a3f8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:15:56.682351 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aabb00e-95a7-4698-a102-1ff22315a3f8 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:15:56.815826 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 611/1205] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 5 15:15:56 2022] POST /identity/v3/users => generated 251 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:15:58.873782 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f0d8ae-9819-4a78-8878-7e9b158aa476 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:15:58.874365 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f0d8ae-9819-4a78-8878-7e9b158aa476 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:15:58.874365 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f0d8ae-9819-4a78-8878-7e9b158aa476 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:15:58.875707 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 595/1206] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:15:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 15:15:58.881089 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07fa80-fef6-4bb4-9362-1a04cb95ce11 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:15:58.881499 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07fa80-fef6-4bb4-9362-1a04cb95ce11 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:15:58.881648 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07fa80-fef6-4bb4-9362-1a04cb95ce11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:15:58.900453 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-9a07fa80-fef6-4bb4-9362-1a04cb95ce11 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:15:58.901647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-9a07fa80-fef6-4bb4-9362-1a04cb95ce11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:15:58.966702 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 612/1207] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:15:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:15:58.990687 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f0d8ae-9819-4a78-8878-7e9b158aa476 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:15:59.094115 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f0d8ae-9819-4a78-8878-7e9b158aa476 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:15:59.129267 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f4651f-1c1a-4734-a01b-6976042c5363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:15:59.129849 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4651f-1c1a-4734-a01b-6976042c5363 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:15:59.130136 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4651f-1c1a-4734-a01b-6976042c5363 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:15:59.130442 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4651f-1c1a-4734-a01b-6976042c5363 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:15:59.140907 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f4651f-1c1a-4734-a01b-6976042c5363 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:15:59.145893 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-d1f4651f-1c1a-4734-a01b-6976042c5363 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:15:59.145893 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:15:59.145893 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:15:59.145893 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:15:59.145893 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:15:59.145893 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:15:59.145893 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:15:59.145893 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:15:59.145893 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:15:59.145893 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:15:59.145893 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:15:59.145893 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:15:59.145893 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:15:59.145893 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:15:59.145893 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:15:59.145893 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:15:59.147060 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:15:59.147060 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:15:59.147060 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:15:59.147060 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:15:59.147060 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:15:59.147060 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:15:59.147060 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:15:59.147060 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:15:59.147060 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:15:59.147060 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:15:59.147060 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:15:59.147060 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:15:59.147060 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:15:59.147060 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:15:59.147060 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:15:59.147060 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:15:59.148057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:15:59.148057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:15:59.148057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:15:59.148057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:15:59.148057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:15:59.148057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:15:59.148057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:15:59.148057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:15:59.148057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:15:59.148057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:15:59.148057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:15:59.148057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:15:59.148057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 596/1208] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 15:15:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:15:59.153247 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a07fa80-fef6-4bb4-9362-1a04cb95ce11 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:15:59.195224 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:15:59.253381 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a07fa80-fef6-4bb4-9362-1a04cb95ce11 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:15:59.286297 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978c83ef-d86d-4b2d-b3bb-90399cdf2c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:15:59.286912 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c83ef-d86d-4b2d-b3bb-90399cdf2c5a None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:15:59.287194 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c83ef-d86d-4b2d-b3bb-90399cdf2c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:15:59.287469 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c83ef-d86d-4b2d-b3bb-90399cdf2c5a None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:15:59.288214 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978c83ef-d86d-4b2d-b3bb-90399cdf2c5a None admin] RBAC: Adding query filter params (name=service) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:15:59.288521 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978c83ef-d86d-4b2d-b3bb-90399cdf2c5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:15:59.289889 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978c83ef-d86d-4b2d-b3bb-90399cdf2c5a None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:15:59.298345 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 613/1209] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 15:15:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:15:59.304250 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f4651f-1c1a-4734-a01b-6976042c5363 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:15:59.354367 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:15:59.403941 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f4651f-1c1a-4734-a01b-6976042c5363 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:15:59.436976 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a0f230-434f-4036-825e-c7eca4421160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:15:59.437610 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0f230-434f-4036-825e-c7eca4421160 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:15:59.437895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0f230-434f-4036-825e-c7eca4421160 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:15:59.438176 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0f230-434f-4036-825e-c7eca4421160 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:15:59.448430 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a0f230-434f-4036-825e-c7eca4421160 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:15:59.449891 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a0f230-434f-4036-825e-c7eca4421160 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:15:59.464005 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-40a0f230-434f-4036-825e-c7eca4421160 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:15:59.464005 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:15:59.464005 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:15:59.464005 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:15:59.464005 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:15:59.464005 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:15:59.464005 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:15:59.464005 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:15:59.464005 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:15:59.464005 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:15:59.464005 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:15:59.464005 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:15:59.464005 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:15:59.464005 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:15:59.464005 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:15:59.465216 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:15:59.465216 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:15:59.465216 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:15:59.465216 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:15:59.465216 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:15:59.465216 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:15:59.465216 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:15:59.465216 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:15:59.465216 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:15:59.465216 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:15:59.465216 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:15:59.465216 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:15:59.465216 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:15:59.465216 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:15:59.465216 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:15:59.465216 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:15:59.465216 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:15:59.466854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:15:59.466854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:15:59.466854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:15:59.466854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:15:59.466854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 597/1210] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:15:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:15:59.471648 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978c83ef-d86d-4b2d-b3bb-90399cdf2c5a None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:15:59.568942 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:15:59.571760 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978c83ef-d86d-4b2d-b3bb-90399cdf2c5a None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:15:59.603884 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d741f8-6cee-4ef1-ad1b-75ca326e153b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:15:59.604527 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d741f8-6cee-4ef1-ad1b-75ca326e153b None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:15:59.604813 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d741f8-6cee-4ef1-ad1b-75ca326e153b None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:15:59.605093 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d741f8-6cee-4ef1-ad1b-75ca326e153b None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:15:59.605779 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d741f8-6cee-4ef1-ad1b-75ca326e153b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:15:59.606160 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d741f8-6cee-4ef1-ad1b-75ca326e153b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:15:59.608101 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d741f8-6cee-4ef1-ad1b-75ca326e153b None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:15:59.619892 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 614/1211] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:15:59.625588 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a0f230-434f-4036-825e-c7eca4421160 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:15:59.727003 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a0f230-434f-4036-825e-c7eca4421160 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:15:59.728493 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:15:59.759444 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2795bf-6382-4d11-91a4-9bf035dd6e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:15:59.761155 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2795bf-6382-4d11-91a4-9bf035dd6e11 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:15:59.761155 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2795bf-6382-4d11-91a4-9bf035dd6e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:15:59.761155 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2795bf-6382-4d11-91a4-9bf035dd6e11 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:15:59.768522 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2795bf-6382-4d11-91a4-9bf035dd6e11 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:15:59.769961 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2795bf-6382-4d11-91a4-9bf035dd6e11 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:15:59.782019 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-dc2795bf-6382-4d11-91a4-9bf035dd6e11 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 05 15:15:59.782019 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:15:59.782019 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:15:59.782019 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:15:59.782019 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:15:59.782019 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:15:59.782019 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:15:59.782019 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:15:59.782019 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:15:59.782019 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:15:59.782019 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:15:59.782019 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:15:59.782019 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 15:15:59.782019 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 15:15:59.782019 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 15:15:59.782019 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 15:15:59.783248 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:15:59.783248 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:15:59.783248 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 15:15:59.783248 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:15:59.783248 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 15:15:59.783248 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:15:59.783248 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:15:59.783248 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:15:59.783248 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:15:59.783248 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:15:59.783248 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:15:59.783248 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:15:59.783248 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:15:59.783248 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:15:59.783248 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:15:59.783248 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:15:59.783248 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:15:59.783248 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:15:59.784537 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:15:59.784537 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:15:59.784537 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 15:15:59.784537 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 15:15:59.784537 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 15:15:59.784537 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 15:15:59.784537 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 15:15:59.784537 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 15:15:59.784537 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 05 15:15:59.784537 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:15:59.784537 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 598/1212] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:15:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:15:59.789516 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d741f8-6cee-4ef1-ad1b-75ca326e153b None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:15:59.888403 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d741f8-6cee-4ef1-ad1b-75ca326e153b None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:15:59.921934 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e305ae-d18c-4233-88d0-d936f1fafd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:15:59.922541 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e305ae-d18c-4233-88d0-d936f1fafd8f None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:15:59.922828 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e305ae-d18c-4233-88d0-d936f1fafd8f None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:15:59.923110 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e305ae-d18c-4233-88d0-d936f1fafd8f None admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:15:59.923900 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e305ae-d18c-4233-88d0-d936f1fafd8f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:15:59.924245 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e305ae-d18c-4233-88d0-d936f1fafd8f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:15:59.925673 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e305ae-d18c-4233-88d0-d936f1fafd8f None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:15:59.942301 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 615/1213] 10.222.0.40 () {56 vars in 1143 bytes} [Fri Aug 5 15:15:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:15:59.948183 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2795bf-6382-4d11-91a4-9bf035dd6e11 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:00.047067 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2795bf-6382-4d11-91a4-9bf035dd6e11 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:00.083569 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc4db3-0e1d-4cb4-92e4-af1d883b2016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:00.084211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc4db3-0e1d-4cb4-92e4-af1d883b2016 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:00.084497 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc4db3-0e1d-4cb4-92e4-af1d883b2016 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:00.084774 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc4db3-0e1d-4cb4-92e4-af1d883b2016 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:00.114272 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:00.151887 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc4db3-0e1d-4cb4-92e4-af1d883b2016 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:00.153381 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc4db3-0e1d-4cb4-92e4-af1d883b2016 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:00.249548 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 599/1214] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:15:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 15:16:00.255426 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e305ae-d18c-4233-88d0-d936f1fafd8f None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:00.274427 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:00.352374 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e305ae-d18c-4233-88d0-d936f1fafd8f None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:00.385665 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa070564-afe8-4eaf-9bca-dbf0a81f201c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:00.386314 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa070564-afe8-4eaf-9bca-dbf0a81f201c None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:00.386581 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa070564-afe8-4eaf-9bca-dbf0a81f201c None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:00.386856 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa070564-afe8-4eaf-9bca-dbf0a81f201c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:00.397227 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa070564-afe8-4eaf-9bca-dbf0a81f201c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:00.398671 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa070564-afe8-4eaf-9bca-dbf0a81f201c None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:00.413537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-aa070564-afe8-4eaf-9bca-dbf0a81f201c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:16:00.413537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:16:00.413537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:16:00.413537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:16:00.413537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:16:00.413537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:16:00.413537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:16:00.413537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:16:00.413537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:16:00.413537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:16:00.413537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:16:00.413537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:16:00.413537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:16:00.413537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:16:00.413537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:16:00.414647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:16:00.414647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:16:00.414647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:16:00.414647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:16:00.414647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:16:00.414647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:16:00.414647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:16:00.414647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:16:00.414647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:16:00.414647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:16:00.414647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:16:00.414647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:16:00.414647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:16:00.414647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:16:00.414647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:16:00.414647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:16:00.414647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:16:00.417000 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:16:00.417000 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:16:00.417000 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:16:00.417000 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:16:00.417000 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 616/1215] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:16:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:16:00.420765 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc4db3-0e1d-4cb4-92e4-af1d883b2016 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:00.481793 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:00.519407 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc4db3-0e1d-4cb4-92e4-af1d883b2016 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:00.552672 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8e7e6-2139-470f-947c-e681e0fcad17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:00.553278 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8e7e6-2139-470f-947c-e681e0fcad17 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:00.553561 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8e7e6-2139-470f-947c-e681e0fcad17 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:00.553837 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8e7e6-2139-470f-947c-e681e0fcad17 None admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:00.554570 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde8e7e6-2139-470f-947c-e681e0fcad17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:16:00.554880 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde8e7e6-2139-470f-947c-e681e0fcad17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:00.556296 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde8e7e6-2139-470f-947c-e681e0fcad17 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:00.567553 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 600/1216] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:16:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:16:00.573461 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa070564-afe8-4eaf-9bca-dbf0a81f201c None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:00.643389 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:00.673602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa070564-afe8-4eaf-9bca-dbf0a81f201c None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:00.709408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d7f7cc-4f71-41cc-a8ac-ddf2017f4467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:00.710045 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d7f7cc-4f71-41cc-a8ac-ddf2017f4467 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:00.710338 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d7f7cc-4f71-41cc-a8ac-ddf2017f4467 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:00.710619 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d7f7cc-4f71-41cc-a8ac-ddf2017f4467 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:00.717240 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d7f7cc-4f71-41cc-a8ac-ddf2017f4467 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:00.718707 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d7f7cc-4f71-41cc-a8ac-ddf2017f4467 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:00.730957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-97d7f7cc-4f71-41cc-a8ac-ddf2017f4467 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:16:00.730957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:16:00.730957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:16:00.730957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:16:00.730957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:16:00.730957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:16:00.730957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:16:00.730957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:16:00.730957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:16:00.730957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:16:00.730957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:16:00.730957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:16:00.730957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 15:16:00.730957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 15:16:00.730957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 15:16:00.735741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 15:16:00.735741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:16:00.735741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:16:00.735741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:16:00.735741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:16:00.735741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:16:00.735741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:16:00.735741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:16:00.735741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:16:00.735741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:16:00.735741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:16:00.735741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:16:00.735741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:16:00.735741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:16:00.735741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:16:00.735741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:16:00.735741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:16:00.736877 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 15:16:00.736877 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 15:16:00.736877 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 15:16:00.736877 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 15:16:00.736877 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 15:16:00.736877 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 15:16:00.736877 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:16:00.736877 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:16:00.736877 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 617/1217] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:16:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:16:00.738685 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8e7e6-2139-470f-947c-e681e0fcad17 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:00.838823 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8e7e6-2139-470f-947c-e681e0fcad17 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:00.861409 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:00.872737 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615222a3-dd6c-488e-8db1-a1932a2372a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:00.873348 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615222a3-dd6c-488e-8db1-a1932a2372a2 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:00.873633 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615222a3-dd6c-488e-8db1-a1932a2372a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:00.873911 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615222a3-dd6c-488e-8db1-a1932a2372a2 None admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:00.874658 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615222a3-dd6c-488e-8db1-a1932a2372a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:16:00.874975 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615222a3-dd6c-488e-8db1-a1932a2372a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:00.876428 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615222a3-dd6c-488e-8db1-a1932a2372a2 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:00.888007 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 601/1218] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 15:16:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:16:00.895667 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d7f7cc-4f71-41cc-a8ac-ddf2017f4467 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:00.993143 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d7f7cc-4f71-41cc-a8ac-ddf2017f4467 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:01.020105 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:01.025583 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c9c1c-ce27-4f4d-bc0d-bf73944ae0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:01.026190 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c9c1c-ce27-4f4d-bc0d-bf73944ae0f0 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:01.026475 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c9c1c-ce27-4f4d-bc0d-bf73944ae0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:01.026755 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c9c1c-ce27-4f4d-bc0d-bf73944ae0f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:01.027538 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c9c1c-ce27-4f4d-bc0d-bf73944ae0f0 None admin] RBAC: Adding query filter params (role.id=266a023f66e043f99efe2941fe58d481, scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83, user.id=107a7c18a2c6440b86e1b7c3e39cc724) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:16:01.027857 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c9c1c-ce27-4f4d-bc0d-bf73944ae0f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:01.029844 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c9c1c-ce27-4f4d-bc0d-bf73944ae0f0 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:01.035107 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 618/1219] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:16:00 2022] GET /identity/v3/role_assignments?user.id=107a7c18a2c6440b86e1b7c3e39cc724&scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83&role.id=266a023f66e043f99efe2941fe58d481 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:16:03.092034 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ffd6c-89f8-48fe-ab3f-1cb2bb5a7f7e None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:03.092620 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ffd6c-89f8-48fe-ab3f-1cb2bb5a7f7e None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:03.092620 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ffd6c-89f8-48fe-ab3f-1cb2bb5a7f7e None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:03.093987 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 602/1220] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:16:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:16:03.099179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e7962-1fe6-4809-9e86-eb54cfdcd8de None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:03.099615 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e7962-1fe6-4809-9e86-eb54cfdcd8de None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:03.099818 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e7962-1fe6-4809-9e86-eb54cfdcd8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:03.117897 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-df9e7962-1fe6-4809-9e86-eb54cfdcd8de None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:16:03.119094 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-df9e7962-1fe6-4809-9e86-eb54cfdcd8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:16:03.185937 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 619/1221] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:16:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:16:03.208863 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ffd6c-89f8-48fe-ab3f-1cb2bb5a7f7e None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:03.308860 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ffd6c-89f8-48fe-ab3f-1cb2bb5a7f7e None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:03.342392 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c49607-d4cd-4db4-92fc-c07d1ad82610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:03.342960 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c49607-d4cd-4db4-92fc-c07d1ad82610 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:03.343246 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c49607-d4cd-4db4-92fc-c07d1ad82610 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:03.343528 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c49607-d4cd-4db4-92fc-c07d1ad82610 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:03.353505 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c49607-d4cd-4db4-92fc-c07d1ad82610 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:16:03.358439 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-61c49607-d4cd-4db4-92fc-c07d1ad82610 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:16:03.358439 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:16:03.358439 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:16:03.358439 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:16:03.358439 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:16:03.358439 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:16:03.358439 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:16:03.358439 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:16:03.358439 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:16:03.358439 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:16:03.358439 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:16:03.358439 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:16:03.358439 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:16:03.358439 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:16:03.358439 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:16:03.358439 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:16:03.359583 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:16:03.359583 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:16:03.359583 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:16:03.359583 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:16:03.359583 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:16:03.359583 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:16:03.359583 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:16:03.359583 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:16:03.359583 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:16:03.359583 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:16:03.359583 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:16:03.359583 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:16:03.359583 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:16:03.359583 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:16:03.359583 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:16:03.359583 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:16:03.360831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:16:03.360831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:16:03.360831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:16:03.360831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:16:03.360831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:16:03.360831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:16:03.360831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:16:03.360831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:16:03.360831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:16:03.360831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:16:03.360831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:16:03.360831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:16:03.360831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 603/1222] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 15:16:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:16:03.365657 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9e7962-1fe6-4809-9e86-eb54cfdcd8de None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:03.463737 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9e7962-1fe6-4809-9e86-eb54cfdcd8de None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:03.487146 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:03.498461 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c088d99c-49a3-4062-a37e-688faf9c6b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:03.499184 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c088d99c-49a3-4062-a37e-688faf9c6b2b None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:03.499465 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c088d99c-49a3-4062-a37e-688faf9c6b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:03.499737 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c088d99c-49a3-4062-a37e-688faf9c6b2b None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:03.500479 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c088d99c-49a3-4062-a37e-688faf9c6b2b None admin] RBAC: Adding query filter params (name=service) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:16:03.500783 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c088d99c-49a3-4062-a37e-688faf9c6b2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:03.502216 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c088d99c-49a3-4062-a37e-688faf9c6b2b None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:03.509855 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 620/1223] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 15:16:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:16:03.515544 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c49607-d4cd-4db4-92fc-c07d1ad82610 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:03.558646 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:03.613731 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c49607-d4cd-4db4-92fc-c07d1ad82610 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:03.646880 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dbcf08-9344-4713-b32e-0ec1d12c7665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:03.647501 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dbcf08-9344-4713-b32e-0ec1d12c7665 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:03.647785 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dbcf08-9344-4713-b32e-0ec1d12c7665 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:03.648087 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dbcf08-9344-4713-b32e-0ec1d12c7665 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:03.657707 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dbcf08-9344-4713-b32e-0ec1d12c7665 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:03.659241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dbcf08-9344-4713-b32e-0ec1d12c7665 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:03.675081 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-39dbcf08-9344-4713-b32e-0ec1d12c7665 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:16:03.675081 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:16:03.675081 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:16:03.675081 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:16:03.675081 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:16:03.675081 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:16:03.675081 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:16:03.675081 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:16:03.675081 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:16:03.675081 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:16:03.675081 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:16:03.675081 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:16:03.675081 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:16:03.675081 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:16:03.675081 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:16:03.676204 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:16:03.676204 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:16:03.676204 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:16:03.676204 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:16:03.676204 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:16:03.676204 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:16:03.676204 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:16:03.676204 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:16:03.676204 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:16:03.676204 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:16:03.676204 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:16:03.676204 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:16:03.676204 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:16:03.676204 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:16:03.676204 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:16:03.676204 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:16:03.676204 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:16:03.677530 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:16:03.677530 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:16:03.677530 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:16:03.677530 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:16:03.677530 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 604/1224] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:16:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:16:03.682050 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c088d99c-49a3-4062-a37e-688faf9c6b2b None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:03.782382 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c088d99c-49a3-4062-a37e-688faf9c6b2b None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:03.815444 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed048e-4313-48da-8be0-266410435393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:03.816051 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed048e-4313-48da-8be0-266410435393 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:03.816363 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed048e-4313-48da-8be0-266410435393 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:03.816637 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed048e-4313-48da-8be0-266410435393 None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:03.817318 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed048e-4313-48da-8be0-266410435393 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:16:03.817623 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed048e-4313-48da-8be0-266410435393 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:03.818979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed048e-4313-48da-8be0-266410435393 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:03.830766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 621/1225] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:16:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:16:03.836826 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dbcf08-9344-4713-b32e-0ec1d12c7665 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:03.929548 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:03.938819 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dbcf08-9344-4713-b32e-0ec1d12c7665 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:03.971775 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60069f0-bf15-44eb-9b8c-423c60770449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:03.972487 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60069f0-bf15-44eb-9b8c-423c60770449 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:03.972779 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60069f0-bf15-44eb-9b8c-423c60770449 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:03.973062 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60069f0-bf15-44eb-9b8c-423c60770449 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:03.980379 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60069f0-bf15-44eb-9b8c-423c60770449 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:03.982399 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60069f0-bf15-44eb-9b8c-423c60770449 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:03.993668 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-e60069f0-bf15-44eb-9b8c-423c60770449 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 05 15:16:03.993668 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:16:03.993668 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:16:03.993668 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:16:03.993668 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:16:03.993668 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:16:03.993668 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:16:03.993668 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:16:03.993668 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:16:03.993668 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:16:03.993668 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:16:03.993668 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:16:03.993668 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 15:16:03.993668 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 15:16:03.993668 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 15:16:03.993668 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 15:16:04.001711 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:16:04.001711 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:16:04.001711 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 15:16:04.001711 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:16:04.001711 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 15:16:04.001711 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:16:04.001711 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:16:04.001711 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:16:04.001711 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:16:04.001711 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:16:04.001711 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:16:04.001711 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:16:04.001711 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:16:04.001711 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:16:04.001711 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:16:04.001711 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:16:04.001711 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:16:04.001711 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:16:04.002909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:16:04.002909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:16:04.002909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 15:16:04.002909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 15:16:04.002909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 15:16:04.002909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 15:16:04.002909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 15:16:04.002909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 15:16:04.002909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 05 15:16:04.002909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:16:04.002909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 605/1226] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:16:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:16:04.003603 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed048e-4313-48da-8be0-266410435393 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:04.047531 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:04.098442 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed048e-4313-48da-8be0-266410435393 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:04.132742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2072a52-42c8-4747-af41-8e26b7865185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:04.133339 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2072a52-42c8-4747-af41-8e26b7865185 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:04.133689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2072a52-42c8-4747-af41-8e26b7865185 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:04.133969 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2072a52-42c8-4747-af41-8e26b7865185 None admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:04.134760 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2072a52-42c8-4747-af41-8e26b7865185 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:16:04.135069 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2072a52-42c8-4747-af41-8e26b7865185 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:04.136543 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2072a52-42c8-4747-af41-8e26b7865185 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:04.153929 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 622/1227] 10.222.0.40 () {56 vars in 1143 bytes} [Fri Aug 5 15:16:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:16:04.159547 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60069f0-bf15-44eb-9b8c-423c60770449 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:04.257663 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60069f0-bf15-44eb-9b8c-423c60770449 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:04.290727 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39640a-b5fe-4b6f-b3f2-f0987c7a273e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:04.291352 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39640a-b5fe-4b6f-b3f2-f0987c7a273e None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:04.291639 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39640a-b5fe-4b6f-b3f2-f0987c7a273e None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:04.291920 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39640a-b5fe-4b6f-b3f2-f0987c7a273e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:04.302397 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a39640a-b5fe-4b6f-b3f2-f0987c7a273e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:04.303834 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a39640a-b5fe-4b6f-b3f2-f0987c7a273e None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:04.318521 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-4a39640a-b5fe-4b6f-b3f2-f0987c7a273e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:16:04.318521 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:16:04.318521 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:16:04.318521 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:16:04.318521 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:16:04.318521 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:16:04.318521 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:16:04.318521 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:16:04.318521 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:16:04.318521 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:16:04.318521 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:16:04.318521 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:16:04.318521 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:16:04.318521 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:16:04.318521 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:16:04.319943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:16:04.319943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:16:04.319943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:16:04.319943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:16:04.319943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:16:04.319943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:16:04.319943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:16:04.319943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:16:04.319943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:16:04.319943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:16:04.319943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:16:04.319943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:16:04.319943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:16:04.319943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:16:04.319943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:16:04.319943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:16:04.319943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:16:04.320909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:16:04.320909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:16:04.320909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:16:04.320909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:16:04.320909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 606/1228] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:16:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:16:04.326145 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2072a52-42c8-4747-af41-8e26b7865185 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:04.424408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2072a52-42c8-4747-af41-8e26b7865185 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:04.426715 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:04.458676 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85754d3e-2f79-473d-92ae-b05cb1e9b45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:04.459288 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85754d3e-2f79-473d-92ae-b05cb1e9b45d None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:04.459605 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85754d3e-2f79-473d-92ae-b05cb1e9b45d None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:04.459881 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85754d3e-2f79-473d-92ae-b05cb1e9b45d None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:04.460599 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85754d3e-2f79-473d-92ae-b05cb1e9b45d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:16:04.460902 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85754d3e-2f79-473d-92ae-b05cb1e9b45d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:04.462263 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85754d3e-2f79-473d-92ae-b05cb1e9b45d None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:04.473967 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 623/1229] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:16:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:16:04.479620 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39640a-b5fe-4b6f-b3f2-f0987c7a273e None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:04.489800 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:04.578890 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39640a-b5fe-4b6f-b3f2-f0987c7a273e None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:04.610841 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e481c-38de-4454-ad28-965c1b6aba96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:04.611474 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e481c-38de-4454-ad28-965c1b6aba96 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:04.611864 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e481c-38de-4454-ad28-965c1b6aba96 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:04.612178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e481c-38de-4454-ad28-965c1b6aba96 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:04.618890 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316e481c-38de-4454-ad28-965c1b6aba96 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:04.620363 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316e481c-38de-4454-ad28-965c1b6aba96 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:04.633043 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-316e481c-38de-4454-ad28-965c1b6aba96 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:16:04.633043 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:16:04.633043 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:16:04.633043 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:16:04.633043 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:16:04.633043 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:16:04.633043 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:16:04.633043 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:16:04.633043 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:16:04.633043 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:16:04.633043 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:16:04.633043 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:16:04.633043 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 15:16:04.633043 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 15:16:04.633043 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 15:16:04.634099 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 15:16:04.634099 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:16:04.634099 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:16:04.634099 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:16:04.634099 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:16:04.634099 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:16:04.634099 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:16:04.634099 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:16:04.634099 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:16:04.634099 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:16:04.634099 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:16:04.634099 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:16:04.634099 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:16:04.634099 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:16:04.634099 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:16:04.634099 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:16:04.634099 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:16:04.635192 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 15:16:04.635192 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 15:16:04.635192 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 15:16:04.635192 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 15:16:04.635192 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 15:16:04.635192 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 15:16:04.635192 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:16:04.635192 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:16:04.635192 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 607/1230] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:16:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:16:04.640665 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85754d3e-2f79-473d-92ae-b05cb1e9b45d None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:04.740076 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85754d3e-2f79-473d-92ae-b05cb1e9b45d None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:04.774697 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d192c-b45f-4f95-9a92-b8f1a0f0f375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:04.775311 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d192c-b45f-4f95-9a92-b8f1a0f0f375 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:04.775596 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d192c-b45f-4f95-9a92-b8f1a0f0f375 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:04.775873 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d192c-b45f-4f95-9a92-b8f1a0f0f375 None admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:04.776654 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484d192c-b45f-4f95-9a92-b8f1a0f0f375 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:16:04.776965 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484d192c-b45f-4f95-9a92-b8f1a0f0f375 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:04.778420 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484d192c-b45f-4f95-9a92-b8f1a0f0f375 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:04.790267 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 624/1231] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 15:16:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:16:04.796433 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e481c-38de-4454-ad28-965c1b6aba96 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:04.860682 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:04.894920 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e481c-38de-4454-ad28-965c1b6aba96 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:04.927660 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d57d9b0-fc77-4e8a-9ec4-1c788b84216c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:04.928367 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d57d9b0-fc77-4e8a-9ec4-1c788b84216c None admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:04.928643 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d57d9b0-fc77-4e8a-9ec4-1c788b84216c None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:04.928921 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d57d9b0-fc77-4e8a-9ec4-1c788b84216c None admin] PATH_INFO: `/v3/projects/06f27ab8886a4c0d81c4d4e3fa100a83/users/107a7c18a2c6440b86e1b7c3e39cc724/roles/266a023f66e043f99efe2941fe58d481` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:04.946086 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d57d9b0-fc77-4e8a-9ec4-1c788b84216c None admin] RBAC: Authorizing `identity:create_grant(project_id=06f27ab8886a4c0d81c4d4e3fa100a83, user_id=107a7c18a2c6440b86e1b7c3e39cc724, role_id=266a023f66e043f99efe2941fe58d481)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:04.947535 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d57d9b0-fc77-4e8a-9ec4-1c788b84216c None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:04.998830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 608/1232] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 15:16:04 2022] PUT /identity/v3/projects/06f27ab8886a4c0d81c4d4e3fa100a83/users/107a7c18a2c6440b86e1b7c3e39cc724/roles/266a023f66e043f99efe2941fe58d481 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:16:07.050276 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1940b-74a1-429d-9014-7ba037b916dc None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:07.050821 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1940b-74a1-429d-9014-7ba037b916dc None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:07.050821 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1940b-74a1-429d-9014-7ba037b916dc None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:07.052185 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 625/1233] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:16:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:16:07.057390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a75c86-3c16-47a7-869e-96ea7410f028 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:07.057768 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a75c86-3c16-47a7-869e-96ea7410f028 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:07.057947 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a75c86-3c16-47a7-869e-96ea7410f028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:07.076747 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-a6a75c86-3c16-47a7-869e-96ea7410f028 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:16:07.077978 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-a6a75c86-3c16-47a7-869e-96ea7410f028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:16:07.143619 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 609/1234] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:16:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:16:07.168916 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b1940b-74a1-429d-9014-7ba037b916dc None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:07.192622 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:07.271222 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b1940b-74a1-429d-9014-7ba037b916dc None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:07.309566 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51299449-790b-47a7-9522-ed84b75e1e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:07.310131 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51299449-790b-47a7-9522-ed84b75e1e23 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:07.310418 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51299449-790b-47a7-9522-ed84b75e1e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:07.310702 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51299449-790b-47a7-9522-ed84b75e1e23 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:07.321813 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51299449-790b-47a7-9522-ed84b75e1e23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:16:07.326726 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-51299449-790b-47a7-9522-ed84b75e1e23 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:16:07.326726 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:16:07.326726 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:16:07.326726 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:16:07.326726 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:16:07.326726 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:16:07.326726 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:16:07.326726 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:16:07.326726 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:16:07.326726 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:16:07.326726 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:16:07.326726 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:16:07.326726 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:16:07.326726 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:16:07.326726 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:16:07.326726 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:16:07.327765 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:16:07.327765 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:16:07.327765 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:16:07.327765 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:16:07.327765 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:16:07.327765 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:16:07.327765 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:16:07.327765 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:16:07.327765 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:16:07.327765 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:16:07.327765 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:16:07.327765 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:16:07.327765 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:16:07.327765 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:16:07.327765 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:16:07.327765 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:16:07.330028 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:16:07.330028 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:16:07.330028 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:16:07.330028 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:16:07.330028 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:16:07.330028 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:16:07.330028 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:16:07.330028 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:16:07.330028 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:16:07.330028 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:16:07.330028 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 15:16:07.330028 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:16:07.330028 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 626/1235] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 5 15:16:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:16:07.333943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a75c86-3c16-47a7-869e-96ea7410f028 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:07.380849 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:07.432082 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a75c86-3c16-47a7-869e-96ea7410f028 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:07.466834 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820814d2-2ab2-486a-878d-eeec00d4e116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:07.467442 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820814d2-2ab2-486a-878d-eeec00d4e116 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:07.467728 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820814d2-2ab2-486a-878d-eeec00d4e116 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:07.468002 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820814d2-2ab2-486a-878d-eeec00d4e116 None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:07.468761 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820814d2-2ab2-486a-878d-eeec00d4e116 None admin] RBAC: Adding query filter params (name=service) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:16:07.469066 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820814d2-2ab2-486a-878d-eeec00d4e116 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:07.470413 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820814d2-2ab2-486a-878d-eeec00d4e116 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:07.478456 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 610/1236] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 5 15:16:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:16:07.484935 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51299449-790b-47a7-9522-ed84b75e1e23 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:07.570997 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:07.589431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51299449-790b-47a7-9522-ed84b75e1e23 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:07.623924 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd1c88-b978-4b02-b5c0-84cac76ee770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:07.624590 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd1c88-b978-4b02-b5c0-84cac76ee770 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:07.624878 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd1c88-b978-4b02-b5c0-84cac76ee770 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:07.625155 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd1c88-b978-4b02-b5c0-84cac76ee770 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:07.634862 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dd1c88-b978-4b02-b5c0-84cac76ee770 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:07.636350 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dd1c88-b978-4b02-b5c0-84cac76ee770 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:07.650517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-b7dd1c88-b978-4b02-b5c0-84cac76ee770 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:16:07.650517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:16:07.650517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:16:07.650517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:16:07.650517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:16:07.650517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:16:07.650517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:16:07.650517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:16:07.650517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:16:07.650517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:16:07.650517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:16:07.650517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:16:07.650517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:16:07.650517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:16:07.650517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:16:07.651666 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:16:07.651666 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:16:07.651666 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:16:07.651666 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:16:07.651666 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:16:07.651666 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:16:07.651666 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:16:07.651666 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:16:07.651666 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:16:07.651666 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:16:07.651666 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:16:07.651666 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:16:07.651666 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:16:07.651666 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:16:07.651666 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:16:07.651666 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:16:07.651666 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:16:07.653431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:16:07.653431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:16:07.653431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:16:07.653431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:16:07.653431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 627/1237] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:16:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 15:16:07.658000 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820814d2-2ab2-486a-878d-eeec00d4e116 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:07.755943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820814d2-2ab2-486a-878d-eeec00d4e116 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:07.764716 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:07.788874 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74815b7f-7857-40c1-8eb0-86c186a15ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:07.789515 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74815b7f-7857-40c1-8eb0-86c186a15ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:07.789796 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74815b7f-7857-40c1-8eb0-86c186a15ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:07.790072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74815b7f-7857-40c1-8eb0-86c186a15ea5 None admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:07.790755 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74815b7f-7857-40c1-8eb0-86c186a15ea5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:16:07.791061 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74815b7f-7857-40c1-8eb0-86c186a15ea5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:07.792447 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74815b7f-7857-40c1-8eb0-86c186a15ea5 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:07.804761 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 611/1238] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:16:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:16:07.810638 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd1c88-b978-4b02-b5c0-84cac76ee770 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:07.908641 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd1c88-b978-4b02-b5c0-84cac76ee770 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:07.941993 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccb3cec-7e00-4bc2-ae76-b42d0922d558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:07.942635 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb3cec-7e00-4bc2-ae76-b42d0922d558 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:07.942949 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb3cec-7e00-4bc2-ae76-b42d0922d558 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:07.943227 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb3cec-7e00-4bc2-ae76-b42d0922d558 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:07.946095 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:07.952507 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccb3cec-7e00-4bc2-ae76-b42d0922d558 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:07.954003 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccb3cec-7e00-4bc2-ae76-b42d0922d558 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:07.965735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-2ccb3cec-7e00-4bc2-ae76-b42d0922d558 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 05 15:16:07.965735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:16:07.965735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:16:07.965735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:16:07.965735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:16:07.965735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:16:07.965735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:16:07.965735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:16:07.965735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:16:07.965735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:16:07.965735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:16:07.965735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:16:07.965735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 15:16:07.965735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 15:16:07.965735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 15:16:07.965735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 15:16:07.966778 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:16:07.966778 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:16:07.966778 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 15:16:07.966778 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:16:07.966778 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 15:16:07.966778 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:16:07.966778 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:16:07.966778 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:16:07.966778 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:16:07.966778 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:16:07.966778 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:16:07.966778 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:16:07.966778 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:16:07.966778 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:16:07.966778 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:16:07.966778 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:16:07.966778 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:16:07.966778 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:16:07.967757 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:16:07.967757 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:16:07.967757 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 15:16:07.967757 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 15:16:07.967757 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 15:16:07.967757 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 15:16:07.967757 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 15:16:07.967757 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 15:16:07.967757 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 05 15:16:07.967757 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:16:07.967757 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 628/1239] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:16:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:16:07.972795 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74815b7f-7857-40c1-8eb0-86c186a15ea5 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:08.071580 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74815b7f-7857-40c1-8eb0-86c186a15ea5 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:08.106195 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46185e-c118-41bb-9ca0-2f32ed871316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:08.106809 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46185e-c118-41bb-9ca0-2f32ed871316 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:08.107124 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46185e-c118-41bb-9ca0-2f32ed871316 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:08.107405 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46185e-c118-41bb-9ca0-2f32ed871316 None admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:08.108230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a46185e-c118-41bb-9ca0-2f32ed871316 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:16:08.108548 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a46185e-c118-41bb-9ca0-2f32ed871316 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:08.109989 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a46185e-c118-41bb-9ca0-2f32ed871316 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:08.126952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 612/1240] 10.222.0.40 () {56 vars in 1143 bytes} [Fri Aug 5 15:16:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:16:08.132942 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccb3cec-7e00-4bc2-ae76-b42d0922d558 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:08.232924 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccb3cec-7e00-4bc2-ae76-b42d0922d558 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:08.267846 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbe3bca-a722-4a11-b090-1191b05da180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:08.268487 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe3bca-a722-4a11-b090-1191b05da180 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:08.268778 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe3bca-a722-4a11-b090-1191b05da180 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:08.269060 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe3bca-a722-4a11-b090-1191b05da180 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:08.329737 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:08.336322 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbe3bca-a722-4a11-b090-1191b05da180 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:08.337780 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbe3bca-a722-4a11-b090-1191b05da180 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:08.434521 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 629/1241] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:16:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 15:16:08.440484 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46185e-c118-41bb-9ca0-2f32ed871316 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:08.467627 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:08.538590 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46185e-c118-41bb-9ca0-2f32ed871316 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:08.571918 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db65d58-8332-43c1-b230-07212f7ef3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:08.572587 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db65d58-8332-43c1-b230-07212f7ef3c1 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:08.572870 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db65d58-8332-43c1-b230-07212f7ef3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:08.573151 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db65d58-8332-43c1-b230-07212f7ef3c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:08.583507 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db65d58-8332-43c1-b230-07212f7ef3c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:08.584986 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db65d58-8332-43c1-b230-07212f7ef3c1 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:08.599240 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-8db65d58-8332-43c1-b230-07212f7ef3c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:16:08.599240 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:16:08.599240 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:16:08.599240 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:16:08.599240 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:16:08.599240 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:16:08.599240 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:16:08.599240 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:16:08.599240 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:16:08.599240 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:16:08.599240 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:16:08.599240 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:16:08.599240 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:16:08.599240 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:16:08.599240 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:16:08.600359 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:16:08.600359 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:16:08.600359 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:16:08.600359 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:16:08.600359 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:16:08.600359 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:16:08.600359 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:16:08.600359 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:16:08.600359 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:16:08.600359 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:16:08.600359 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:16:08.600359 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:16:08.600359 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:16:08.600359 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:16:08.600359 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:16:08.600359 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:16:08.600359 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:16:08.601461 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:16:08.601461 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:16:08.601461 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:16:08.601461 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:16:08.601461 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 613/1242] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:16:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:16:08.606390 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbe3bca-a722-4a11-b090-1191b05da180 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:08.712965 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:08.712965 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbe3bca-a722-4a11-b090-1191b05da180 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:08.744330 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d9c2e-e7f9-420a-97e3-5cb5bc6549c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:08.749302 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d9c2e-e7f9-420a-97e3-5cb5bc6549c9 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:08.749302 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d9c2e-e7f9-420a-97e3-5cb5bc6549c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:08.749302 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d9c2e-e7f9-420a-97e3-5cb5bc6549c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:08.749302 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0d9c2e-e7f9-420a-97e3-5cb5bc6549c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:16:08.749302 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0d9c2e-e7f9-420a-97e3-5cb5bc6549c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:08.749302 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0d9c2e-e7f9-420a-97e3-5cb5bc6549c9 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:08.759931 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 630/1243] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:16:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:16:08.765533 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db65d58-8332-43c1-b230-07212f7ef3c1 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:08.844328 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:08.865679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db65d58-8332-43c1-b230-07212f7ef3c1 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:08.898986 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc99e21-b1f3-4078-b4e5-445c74f5ac10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:08.899623 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc99e21-b1f3-4078-b4e5-445c74f5ac10 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:08.899912 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc99e21-b1f3-4078-b4e5-445c74f5ac10 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:08.900227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc99e21-b1f3-4078-b4e5-445c74f5ac10 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:08.906758 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc99e21-b1f3-4078-b4e5-445c74f5ac10 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:08.908225 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc99e21-b1f3-4078-b4e5-445c74f5ac10 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:08.920298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-ddc99e21-b1f3-4078-b4e5-445c74f5ac10 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:16:08.920298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:16:08.920298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:16:08.920298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:16:08.920298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:16:08.920298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:16:08.920298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:16:08.920298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:16:08.920298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:16:08.920298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:16:08.920298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:16:08.920298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:16:08.920298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 15:16:08.920298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 15:16:08.920298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 15:16:08.921347 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 15:16:08.921347 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:16:08.921347 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:16:08.921347 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:16:08.921347 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:16:08.921347 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:16:08.921347 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:16:08.921347 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:16:08.921347 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:16:08.921347 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:16:08.921347 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:16:08.921347 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:16:08.921347 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:16:08.921347 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:16:08.921347 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:16:08.921347 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:16:08.921347 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:16:08.922268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 15:16:08.922268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 15:16:08.922268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 15:16:08.922268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 15:16:08.922268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 15:16:08.922268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 15:16:08.922268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:16:08.922268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:16:08.922268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 614/1244] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:16:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:16:08.927464 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d9c2e-e7f9-420a-97e3-5cb5bc6549c9 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:09.026452 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d9c2e-e7f9-420a-97e3-5cb5bc6549c9 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:09.059482 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff531d-1818-4d41-815f-0d979189a6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:09.060130 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff531d-1818-4d41-815f-0d979189a6d3 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:09.060421 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff531d-1818-4d41-815f-0d979189a6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:09.060700 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff531d-1818-4d41-815f-0d979189a6d3 None admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:09.061447 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff531d-1818-4d41-815f-0d979189a6d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:16:09.061759 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff531d-1818-4d41-815f-0d979189a6d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:09.063185 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff531d-1818-4d41-815f-0d979189a6d3 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:09.074917 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 631/1245] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 15:16:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:16:09.081091 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc99e21-b1f3-4078-b4e5-445c74f5ac10 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:09.182942 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc99e21-b1f3-4078-b4e5-445c74f5ac10 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:09.217097 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f6e1e-ad5d-43f2-a296-eb7b9bba8e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:09.217705 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f6e1e-ad5d-43f2-a296-eb7b9bba8e44 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:09.217992 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f6e1e-ad5d-43f2-a296-eb7b9bba8e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:09.218275 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f6e1e-ad5d-43f2-a296-eb7b9bba8e44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:09.219060 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959f6e1e-ad5d-43f2-a296-eb7b9bba8e44 None admin] RBAC: Adding query filter params (role.id=266a023f66e043f99efe2941fe58d481, scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83, user.id=107a7c18a2c6440b86e1b7c3e39cc724) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:16:09.219379 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959f6e1e-ad5d-43f2-a296-eb7b9bba8e44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:09.220843 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959f6e1e-ad5d-43f2-a296-eb7b9bba8e44 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:09.226574 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 615/1246] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:16:09 2022] GET /identity/v3/role_assignments?user.id=107a7c18a2c6440b86e1b7c3e39cc724&scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83&role.id=266a023f66e043f99efe2941fe58d481 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:16:11.250720 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909f297-3005-4a4f-9b36-ebd5e52b12ba None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:11.251239 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909f297-3005-4a4f-9b36-ebd5e52b12ba None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:11.251353 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909f297-3005-4a4f-9b36-ebd5e52b12ba None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:11.253153 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 632/1247] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:16:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:16:11.258730 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b9132-ceb7-4bfc-a82e-a7b5983ac27c None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:11.259110 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b9132-ceb7-4bfc-a82e-a7b5983ac27c None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:11.259288 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b9132-ceb7-4bfc-a82e-a7b5983ac27c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:11.266343 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:11.279513 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-b52b9132-ceb7-4bfc-a82e-a7b5983ac27c None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:16:11.280739 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-b52b9132-ceb7-4bfc-a82e-a7b5983ac27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:16:11.345047 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 616/1248] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:16:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:16:11.368125 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0909f297-3005-4a4f-9b36-ebd5e52b12ba None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:11.386669 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:11.469319 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0909f297-3005-4a4f-9b36-ebd5e52b12ba None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:11.503653 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc457ed-00e0-4388-abcb-1cfd8f5b13e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:11.504320 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc457ed-00e0-4388-abcb-1cfd8f5b13e5 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:11.504607 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc457ed-00e0-4388-abcb-1cfd8f5b13e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:11.504887 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc457ed-00e0-4388-abcb-1cfd8f5b13e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:11.515417 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc457ed-00e0-4388-abcb-1cfd8f5b13e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:16:11.520348 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-7fc457ed-00e0-4388-abcb-1cfd8f5b13e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:16:11.520348 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:16:11.520348 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:16:11.520348 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:16:11.520348 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:16:11.520348 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:16:11.520348 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:16:11.520348 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:16:11.520348 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:16:11.520348 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:16:11.520348 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:16:11.520348 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:16:11.520348 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:16:11.520348 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:16:11.520348 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:16:11.520348 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:16:11.521452 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:16:11.521452 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:16:11.521452 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:16:11.521452 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:16:11.521452 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:16:11.521452 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:16:11.521452 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:16:11.521452 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:16:11.521452 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:16:11.521452 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:16:11.521452 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:16:11.521452 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:16:11.521452 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:16:11.521452 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:16:11.521452 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:16:11.521452 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:16:11.522497 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:16:11.522497 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:16:11.522497 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:16:11.522497 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:16:11.522497 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:16:11.522497 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:16:11.522497 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:16:11.522497 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:16:11.522497 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:16:11.522497 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:16:11.522497 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:16:11.522497 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:16:11.522497 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 633/1249] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 15:16:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:16:11.527827 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b9132-ceb7-4bfc-a82e-a7b5983ac27c None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:11.626050 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b9132-ceb7-4bfc-a82e-a7b5983ac27c None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:11.655214 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:11.660274 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885eadf3-0d0f-4519-a1ee-6c24edd3f629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:11.660902 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885eadf3-0d0f-4519-a1ee-6c24edd3f629 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:11.661183 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885eadf3-0d0f-4519-a1ee-6c24edd3f629 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:11.661460 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885eadf3-0d0f-4519-a1ee-6c24edd3f629 None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:11.662166 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885eadf3-0d0f-4519-a1ee-6c24edd3f629 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:16:11.662471 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885eadf3-0d0f-4519-a1ee-6c24edd3f629 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:11.663845 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885eadf3-0d0f-4519-a1ee-6c24edd3f629 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:11.672638 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 617/1250] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:16:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:16:11.678540 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc457ed-00e0-4388-abcb-1cfd8f5b13e5 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:11.756722 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:11.779391 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc457ed-00e0-4388-abcb-1cfd8f5b13e5 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:11.812656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de45ec-e89e-49cd-b64d-4f82119a7a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:11.813290 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de45ec-e89e-49cd-b64d-4f82119a7a0d None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:11.813576 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de45ec-e89e-49cd-b64d-4f82119a7a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:11.813858 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de45ec-e89e-49cd-b64d-4f82119a7a0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:11.824709 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9de45ec-e89e-49cd-b64d-4f82119a7a0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:11.826165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9de45ec-e89e-49cd-b64d-4f82119a7a0d None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:11.839903 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-b9de45ec-e89e-49cd-b64d-4f82119a7a0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:16:11.839903 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:16:11.839903 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:16:11.839903 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:16:11.839903 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:16:11.839903 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:16:11.839903 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:16:11.839903 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:16:11.839903 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:16:11.839903 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:16:11.839903 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:16:11.839903 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:16:11.839903 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:16:11.839903 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:16:11.839903 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:16:11.841004 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:16:11.841004 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:16:11.841004 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:16:11.841004 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:16:11.841004 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:16:11.841004 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:16:11.841004 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:16:11.841004 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:16:11.841004 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:16:11.841004 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:16:11.841004 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:16:11.841004 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:16:11.841004 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:16:11.841004 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:16:11.841004 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:16:11.841004 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:16:11.841004 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:16:11.842134 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:16:11.842134 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:16:11.842134 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:16:11.842134 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:16:11.842134 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 634/1251] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:16:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:16:11.847184 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885eadf3-0d0f-4519-a1ee-6c24edd3f629 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:11.946189 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885eadf3-0d0f-4519-a1ee-6c24edd3f629 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:11.978479 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a036b5f-c25f-4d8e-84e4-240ca4cd0763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:11.979085 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a036b5f-c25f-4d8e-84e4-240ca4cd0763 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:11.979371 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a036b5f-c25f-4d8e-84e4-240ca4cd0763 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:11.979649 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a036b5f-c25f-4d8e-84e4-240ca4cd0763 None admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:11.980372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a036b5f-c25f-4d8e-84e4-240ca4cd0763 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:16:11.981198 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a036b5f-c25f-4d8e-84e4-240ca4cd0763 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:11.982601 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a036b5f-c25f-4d8e-84e4-240ca4cd0763 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:11.994483 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 618/1252] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:16:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:16:12.000930 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de45ec-e89e-49cd-b64d-4f82119a7a0d None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:12.104292 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de45ec-e89e-49cd-b64d-4f82119a7a0d None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:12.130238 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:12.137939 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d3b65-4e3d-4945-b0ca-7a51c40f826a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:12.138576 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d3b65-4e3d-4945-b0ca-7a51c40f826a None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:12.138859 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d3b65-4e3d-4945-b0ca-7a51c40f826a None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:12.139132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d3b65-4e3d-4945-b0ca-7a51c40f826a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:12.146388 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345d3b65-4e3d-4945-b0ca-7a51c40f826a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:12.147823 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345d3b65-4e3d-4945-b0ca-7a51c40f826a None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:12.159433 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-345d3b65-4e3d-4945-b0ca-7a51c40f826a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 05 15:16:12.159433 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:16:12.159433 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:16:12.159433 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:16:12.159433 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:16:12.159433 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:16:12.159433 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:16:12.159433 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:16:12.159433 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:16:12.159433 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:16:12.159433 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:16:12.159433 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:16:12.159433 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 15:16:12.159433 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 15:16:12.159433 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 15:16:12.159433 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 15:16:12.160660 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:16:12.160660 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:16:12.160660 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 15:16:12.160660 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:16:12.160660 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 15:16:12.160660 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:16:12.160660 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:16:12.160660 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:16:12.160660 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:16:12.160660 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:16:12.160660 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:16:12.160660 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:16:12.160660 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:16:12.160660 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:16:12.160660 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:16:12.160660 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:16:12.160660 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:16:12.160660 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:16:12.161622 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:16:12.161622 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:16:12.161622 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 15:16:12.161622 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 15:16:12.161622 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 15:16:12.161622 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 15:16:12.161622 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 15:16:12.161622 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 15:16:12.161622 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 05 15:16:12.161622 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:16:12.161622 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 635/1253] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:16:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:16:12.166795 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a036b5f-c25f-4d8e-84e4-240ca4cd0763 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:12.221569 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:12.269124 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a036b5f-c25f-4d8e-84e4-240ca4cd0763 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:12.305173 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcbe322-ca74-48ce-bf8a-1f2cc9ca6869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:12.305788 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbe322-ca74-48ce-bf8a-1f2cc9ca6869 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:12.306069 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbe322-ca74-48ce-bf8a-1f2cc9ca6869 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:12.306343 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbe322-ca74-48ce-bf8a-1f2cc9ca6869 None admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:12.307130 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcbe322-ca74-48ce-bf8a-1f2cc9ca6869 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:16:12.307462 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcbe322-ca74-48ce-bf8a-1f2cc9ca6869 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:12.308905 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcbe322-ca74-48ce-bf8a-1f2cc9ca6869 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:12.325571 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 619/1254] 10.222.0.40 () {56 vars in 1143 bytes} [Fri Aug 5 15:16:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:16:12.331519 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d3b65-4e3d-4945-b0ca-7a51c40f826a None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:12.430445 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d3b65-4e3d-4945-b0ca-7a51c40f826a None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:12.463987 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a2a347-5cf8-4447-b8b3-44754386bc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:12.464637 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2a347-5cf8-4447-b8b3-44754386bc78 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:12.464922 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2a347-5cf8-4447-b8b3-44754386bc78 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:12.465787 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2a347-5cf8-4447-b8b3-44754386bc78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:12.519547 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:12.532333 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a2a347-5cf8-4447-b8b3-44754386bc78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:12.537116 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a2a347-5cf8-4447-b8b3-44754386bc78 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:12.629515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 636/1255] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:16:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 15:16:12.635719 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcbe322-ca74-48ce-bf8a-1f2cc9ca6869 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:12.733191 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcbe322-ca74-48ce-bf8a-1f2cc9ca6869 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:12.746754 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:12.767903 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ecfe3d-5e23-45fb-b5a3-5341d0e5ab0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:12.768579 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ecfe3d-5e23-45fb-b5a3-5341d0e5ab0d None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:12.768844 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ecfe3d-5e23-45fb-b5a3-5341d0e5ab0d None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:12.769118 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ecfe3d-5e23-45fb-b5a3-5341d0e5ab0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:12.779637 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ecfe3d-5e23-45fb-b5a3-5341d0e5ab0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:12.781238 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ecfe3d-5e23-45fb-b5a3-5341d0e5ab0d None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:12.796578 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-69ecfe3d-5e23-45fb-b5a3-5341d0e5ab0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:16:12.796578 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:16:12.796578 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:16:12.796578 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:16:12.796578 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:16:12.796578 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:16:12.796578 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:16:12.796578 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:16:12.796578 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:16:12.796578 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:16:12.796578 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:16:12.796578 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:16:12.796578 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:16:12.796578 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:16:12.796578 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:16:12.797722 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:16:12.797722 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:16:12.797722 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:16:12.797722 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:16:12.797722 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:16:12.797722 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:16:12.797722 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:16:12.797722 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:16:12.797722 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:16:12.797722 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:16:12.797722 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:16:12.797722 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:16:12.797722 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:16:12.797722 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:16:12.797722 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:16:12.797722 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:16:12.797722 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:16:12.798898 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:16:12.798898 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:16:12.798898 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:16:12.798898 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:16:12.798898 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 620/1256] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:16:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:16:12.803761 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a2a347-5cf8-4447-b8b3-44754386bc78 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:12.891673 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:12.903375 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a2a347-5cf8-4447-b8b3-44754386bc78 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:12.936628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ea790d-ac87-48cc-aa9f-d95c726ec913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:12.937267 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea790d-ac87-48cc-aa9f-d95c726ec913 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:12.937549 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea790d-ac87-48cc-aa9f-d95c726ec913 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:12.937820 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea790d-ac87-48cc-aa9f-d95c726ec913 None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:12.938503 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ea790d-ac87-48cc-aa9f-d95c726ec913 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:16:12.938807 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ea790d-ac87-48cc-aa9f-d95c726ec913 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:12.940198 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ea790d-ac87-48cc-aa9f-d95c726ec913 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:12.951873 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 637/1257] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:16:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:16:12.957716 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ecfe3d-5e23-45fb-b5a3-5341d0e5ab0d None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:13.055455 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ecfe3d-5e23-45fb-b5a3-5341d0e5ab0d None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:13.088650 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cbe2c6-24d1-4ea3-a9e2-37065f14ef55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:13.089270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cbe2c6-24d1-4ea3-a9e2-37065f14ef55 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:13.089556 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cbe2c6-24d1-4ea3-a9e2-37065f14ef55 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:13.089833 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cbe2c6-24d1-4ea3-a9e2-37065f14ef55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:13.096437 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cbe2c6-24d1-4ea3-a9e2-37065f14ef55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:13.097873 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cbe2c6-24d1-4ea3-a9e2-37065f14ef55 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:13.110867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-a6cbe2c6-24d1-4ea3-a9e2-37065f14ef55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:16:13.110867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:16:13.110867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:16:13.110867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:16:13.110867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:16:13.110867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:16:13.110867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:16:13.110867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:16:13.110867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:16:13.110867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:16:13.110867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:16:13.110867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:16:13.110867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 15:16:13.110867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 15:16:13.110867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 15:16:13.116402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 15:16:13.116402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:16:13.116402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:16:13.116402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:16:13.116402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:16:13.116402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:16:13.116402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:16:13.116402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:16:13.116402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:16:13.116402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:16:13.116402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:16:13.116402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:16:13.116402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:16:13.116402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:16:13.116402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:16:13.116402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:16:13.116402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:16:13.117782 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 15:16:13.117782 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 15:16:13.117782 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 15:16:13.117782 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 15:16:13.117782 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 15:16:13.117782 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 15:16:13.117782 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:16:13.117782 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:16:13.117782 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 621/1258] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:16:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:16:13.118465 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ea790d-ac87-48cc-aa9f-d95c726ec913 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:13.219581 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ea790d-ac87-48cc-aa9f-d95c726ec913 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:13.253065 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e60791-ab97-4e22-9601-aabda3409db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:13.253680 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e60791-ab97-4e22-9601-aabda3409db1 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:13.253967 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e60791-ab97-4e22-9601-aabda3409db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:13.254248 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e60791-ab97-4e22-9601-aabda3409db1 None admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:13.255053 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e60791-ab97-4e22-9601-aabda3409db1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:16:13.255372 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e60791-ab97-4e22-9601-aabda3409db1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:13.256835 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e60791-ab97-4e22-9601-aabda3409db1 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:13.268526 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 638/1259] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 15:16:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:16:13.276203 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cbe2c6-24d1-4ea3-a9e2-37065f14ef55 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:13.301957 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:13.380628 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cbe2c6-24d1-4ea3-a9e2-37065f14ef55 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:13.412616 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da026f4-b4c4-45ee-bd09-5ec5e34060aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:13.413222 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da026f4-b4c4-45ee-bd09-5ec5e34060aa None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:13.413509 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da026f4-b4c4-45ee-bd09-5ec5e34060aa None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:13.413786 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da026f4-b4c4-45ee-bd09-5ec5e34060aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:13.414573 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da026f4-b4c4-45ee-bd09-5ec5e34060aa None admin] RBAC: Adding query filter params (role.id=7c88c815db854954b516ab97cd471647, scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83, user.id=107a7c18a2c6440b86e1b7c3e39cc724) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:16:13.414882 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da026f4-b4c4-45ee-bd09-5ec5e34060aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:13.416314 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da026f4-b4c4-45ee-bd09-5ec5e34060aa None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:13.422177 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 622/1260] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:16:13 2022] GET /identity/v3/role_assignments?user.id=107a7c18a2c6440b86e1b7c3e39cc724&scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83&role.id=7c88c815db854954b516ab97cd471647 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:16:15.369370 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca701e-6a40-4961-bc0b-653ef440bb5b None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:15.369948 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca701e-6a40-4961-bc0b-653ef440bb5b None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:15.369948 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca701e-6a40-4961-bc0b-653ef440bb5b None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:15.371264 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 639/1261] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:16:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:16:15.376563 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0481a-4da4-4b2a-9831-e8c870743339 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:15.377022 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0481a-4da4-4b2a-9831-e8c870743339 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:15.377123 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0481a-4da4-4b2a-9831-e8c870743339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:15.394922 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-f1e0481a-4da4-4b2a-9831-e8c870743339 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:16:15.396156 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-f1e0481a-4da4-4b2a-9831-e8c870743339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:16:15.462185 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 623/1262] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:16:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:16:15.484791 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca701e-6a40-4961-bc0b-653ef440bb5b None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:15.495872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:15.584808 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca701e-6a40-4961-bc0b-653ef440bb5b None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:15.618216 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f73618d-6c7c-43b0-b60f-f5651e85c5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:15.618846 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f73618d-6c7c-43b0-b60f-f5651e85c5c8 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:15.619127 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f73618d-6c7c-43b0-b60f-f5651e85c5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:15.619464 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f73618d-6c7c-43b0-b60f-f5651e85c5c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:15.630533 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f73618d-6c7c-43b0-b60f-f5651e85c5c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:16:15.635554 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-8f73618d-6c7c-43b0-b60f-f5651e85c5c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:16:15.635554 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:16:15.635554 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:16:15.635554 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:16:15.635554 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:16:15.635554 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:16:15.635554 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:16:15.635554 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:16:15.635554 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:16:15.635554 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:16:15.635554 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:16:15.635554 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:16:15.635554 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:16:15.635554 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:16:15.635554 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:16:15.635554 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:16:15.637431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:16:15.637431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:16:15.637431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:16:15.637431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:16:15.637431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:16:15.637431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:16:15.637431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:16:15.637431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:16:15.637431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:16:15.637431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:16:15.637431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:16:15.637431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:16:15.637431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:16:15.637431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:16:15.637431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:16:15.637431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:16:15.638562 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:16:15.638562 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:16:15.638562 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:16:15.638562 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:16:15.638562 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:16:15.638562 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:16:15.638562 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:16:15.638562 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:16:15.638562 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:16:15.638562 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:16:15.638562 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:16:15.638562 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:16:15.638562 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 640/1263] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 15:16:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:16:15.643375 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e0481a-4da4-4b2a-9831-e8c870743339 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:15.661927 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:15.740557 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e0481a-4da4-4b2a-9831-e8c870743339 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:15.774464 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e447fb03-6d92-4e31-b08d-67c5d08a958f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:15.775065 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447fb03-6d92-4e31-b08d-67c5d08a958f None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:15.775345 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447fb03-6d92-4e31-b08d-67c5d08a958f None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:15.775618 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447fb03-6d92-4e31-b08d-67c5d08a958f None admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:15.776344 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e447fb03-6d92-4e31-b08d-67c5d08a958f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:16:15.776649 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e447fb03-6d92-4e31-b08d-67c5d08a958f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:15.778695 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e447fb03-6d92-4e31-b08d-67c5d08a958f None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:15.786530 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 624/1264] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:16:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:16:15.792408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f73618d-6c7c-43b0-b60f-f5651e85c5c8 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:15.859559 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:15.892678 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f73618d-6c7c-43b0-b60f-f5651e85c5c8 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:15.925586 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97bce20-d795-4533-b73c-80614beb96e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:15.926240 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97bce20-d795-4533-b73c-80614beb96e3 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:15.926506 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97bce20-d795-4533-b73c-80614beb96e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:15.926783 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97bce20-d795-4533-b73c-80614beb96e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:15.938630 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97bce20-d795-4533-b73c-80614beb96e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:15.940143 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97bce20-d795-4533-b73c-80614beb96e3 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:15.955074 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-e97bce20-d795-4533-b73c-80614beb96e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:16:15.955074 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:16:15.955074 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:16:15.955074 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:16:15.955074 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:16:15.955074 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:16:15.955074 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:16:15.955074 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:16:15.955074 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:16:15.955074 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:16:15.955074 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:16:15.955074 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:16:15.955074 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:16:15.955074 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:16:15.955074 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:16:15.956132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:16:15.956132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:16:15.956132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:16:15.956132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:16:15.956132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:16:15.956132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:16:15.956132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:16:15.956132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:16:15.956132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:16:15.956132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:16:15.956132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:16:15.956132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:16:15.956132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:16:15.956132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:16:15.956132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:16:15.956132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:16:15.956132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:16:15.957104 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:16:15.957104 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:16:15.957104 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:16:15.957104 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:16:15.957104 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 641/1265] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:16:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:16:15.962149 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e447fb03-6d92-4e31-b08d-67c5d08a958f None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:16.041279 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:16.060397 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e447fb03-6d92-4e31-b08d-67c5d08a958f None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:16.092972 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ed736a-90e7-4ea4-b7a2-c2f5ddde31c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:16.093578 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed736a-90e7-4ea4-b7a2-c2f5ddde31c7 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:16.093862 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed736a-90e7-4ea4-b7a2-c2f5ddde31c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:16.094143 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed736a-90e7-4ea4-b7a2-c2f5ddde31c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:16.094826 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ed736a-90e7-4ea4-b7a2-c2f5ddde31c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:16:16.095125 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ed736a-90e7-4ea4-b7a2-c2f5ddde31c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:16.096505 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ed736a-90e7-4ea4-b7a2-c2f5ddde31c7 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:16.108479 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 625/1266] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:16:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:16:16.114229 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97bce20-d795-4533-b73c-80614beb96e3 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:16.216219 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97bce20-d795-4533-b73c-80614beb96e3 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:16.235757 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:16.250282 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1810d7-8f34-44fa-b840-2e74e9231ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:16.250916 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1810d7-8f34-44fa-b840-2e74e9231ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:16.251202 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1810d7-8f34-44fa-b840-2e74e9231ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:16.251481 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1810d7-8f34-44fa-b840-2e74e9231ad0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:16.259531 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1810d7-8f34-44fa-b840-2e74e9231ad0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:16.261015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1810d7-8f34-44fa-b840-2e74e9231ad0 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:16.272035 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-5f1810d7-8f34-44fa-b840-2e74e9231ad0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 05 15:16:16.272035 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:16:16.272035 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:16:16.272035 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:16:16.272035 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:16:16.272035 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:16:16.272035 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:16:16.272035 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:16:16.272035 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:16:16.272035 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:16:16.272035 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:16:16.272035 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:16:16.272035 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 15:16:16.272035 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 15:16:16.272035 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 15:16:16.272035 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 15:16:16.276619 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:16:16.276619 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:16:16.276619 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 15:16:16.276619 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:16:16.276619 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 15:16:16.276619 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:16:16.276619 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:16:16.276619 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:16:16.276619 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:16:16.276619 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:16:16.276619 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:16:16.276619 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:16:16.276619 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:16:16.276619 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:16:16.276619 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:16:16.276619 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:16:16.276619 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:16:16.276619 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:16:16.277830 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:16:16.277830 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:16:16.277830 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 15:16:16.277830 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 15:16:16.277830 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 15:16:16.277830 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 15:16:16.277830 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 15:16:16.277830 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 15:16:16.277830 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 05 15:16:16.277830 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:16:16.277830 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 642/1267] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:16:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:16:16.279125 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ed736a-90e7-4ea4-b7a2-c2f5ddde31c7 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:16.379493 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ed736a-90e7-4ea4-b7a2-c2f5ddde31c7 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:16.412139 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6df772b-82b4-4bb2-8675-939b0ca1ef73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:16.412777 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6df772b-82b4-4bb2-8675-939b0ca1ef73 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:16.413060 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6df772b-82b4-4bb2-8675-939b0ca1ef73 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:16.413329 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6df772b-82b4-4bb2-8675-939b0ca1ef73 None admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:16.414105 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6df772b-82b4-4bb2-8675-939b0ca1ef73 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:16:16.414410 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6df772b-82b4-4bb2-8675-939b0ca1ef73 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:16.415827 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6df772b-82b4-4bb2-8675-939b0ca1ef73 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:16.424814 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:16.433774 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 626/1268] 10.222.0.40 () {56 vars in 1143 bytes} [Fri Aug 5 15:16:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:16:16.440342 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1810d7-8f34-44fa-b840-2e74e9231ad0 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:16.538104 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1810d7-8f34-44fa-b840-2e74e9231ad0 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:16.571628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887db982-272f-4fce-9141-fee945ea7a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:16.572282 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887db982-272f-4fce-9141-fee945ea7a0f None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:16.572567 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887db982-272f-4fce-9141-fee945ea7a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:16.572846 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887db982-272f-4fce-9141-fee945ea7a0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:16.583103 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887db982-272f-4fce-9141-fee945ea7a0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:16.584575 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887db982-272f-4fce-9141-fee945ea7a0f None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:16.598982 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-887db982-272f-4fce-9141-fee945ea7a0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:16:16.598982 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:16:16.598982 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:16:16.598982 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:16:16.598982 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:16:16.598982 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:16:16.598982 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:16:16.598982 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:16:16.598982 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:16:16.598982 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:16:16.598982 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:16:16.598982 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:16:16.598982 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:16:16.598982 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:16:16.598982 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:16:16.602628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:16:16.602628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:16:16.602628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:16:16.602628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:16:16.602628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:16:16.602628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:16:16.602628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:16:16.602628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:16:16.602628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:16:16.602628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:16:16.602628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:16:16.602628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:16:16.602628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:16:16.602628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:16:16.602628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:16:16.602628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:16:16.602628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:16:16.603770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:16:16.603770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:16:16.603770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:16:16.603770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:16:16.603770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 643/1269] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:16:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:16:16.606812 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6df772b-82b4-4bb2-8675-939b0ca1ef73 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:16.708937 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6df772b-82b4-4bb2-8675-939b0ca1ef73 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:16.743728 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2fc372-8efa-4ec9-a086-d539d00be9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:16.744354 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2fc372-8efa-4ec9-a086-d539d00be9ea None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:16.744637 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2fc372-8efa-4ec9-a086-d539d00be9ea None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:16.744914 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2fc372-8efa-4ec9-a086-d539d00be9ea None admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:16.745595 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2fc372-8efa-4ec9-a086-d539d00be9ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:16:16.745961 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2fc372-8efa-4ec9-a086-d539d00be9ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:16.747467 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2fc372-8efa-4ec9-a086-d539d00be9ea None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:16.761879 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 627/1270] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:16:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:16:16.767674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887db982-272f-4fce-9141-fee945ea7a0f None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:16.800212 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:16.868832 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887db982-272f-4fce-9141-fee945ea7a0f None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:16.901466 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd9c4b-b0f3-4c0b-a5a5-1e7dd6121b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:16.902101 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd9c4b-b0f3-4c0b-a5a5-1e7dd6121b7b None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:16.902387 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd9c4b-b0f3-4c0b-a5a5-1e7dd6121b7b None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:16.903202 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd9c4b-b0f3-4c0b-a5a5-1e7dd6121b7b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:16.909802 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dd9c4b-b0f3-4c0b-a5a5-1e7dd6121b7b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:16.911252 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dd9c4b-b0f3-4c0b-a5a5-1e7dd6121b7b None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:16.925426 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-90dd9c4b-b0f3-4c0b-a5a5-1e7dd6121b7b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:16:16.925426 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:16:16.925426 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:16:16.925426 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:16:16.925426 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:16:16.925426 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:16:16.925426 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:16:16.925426 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:16:16.925426 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:16:16.925426 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:16:16.925426 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:16:16.925426 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:16:16.925426 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 15:16:16.925426 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 15:16:16.925426 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 15:16:16.926531 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 15:16:16.926531 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:16:16.926531 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:16:16.926531 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:16:16.926531 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:16:16.926531 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:16:16.926531 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:16:16.926531 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:16:16.926531 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:16:16.926531 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:16:16.926531 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:16:16.926531 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:16:16.926531 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:16:16.926531 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:16:16.926531 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:16:16.926531 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:16:16.926531 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:16:16.932324 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 15:16:16.932324 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 15:16:16.932324 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 15:16:16.932324 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 15:16:16.932324 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 15:16:16.932324 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 15:16:16.932324 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:16:16.932324 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:16:16.932324 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 644/1271] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:16:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:16:16.932962 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2fc372-8efa-4ec9-a086-d539d00be9ea None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:16.992293 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:17.034194 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2fc372-8efa-4ec9-a086-d539d00be9ea None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:17.069471 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057988ac-9542-41ac-8fd2-75880d156f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:17.070084 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057988ac-9542-41ac-8fd2-75880d156f41 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:17.070373 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057988ac-9542-41ac-8fd2-75880d156f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:17.070654 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057988ac-9542-41ac-8fd2-75880d156f41 None admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:17.071427 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057988ac-9542-41ac-8fd2-75880d156f41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:16:17.071736 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057988ac-9542-41ac-8fd2-75880d156f41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:17.073231 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057988ac-9542-41ac-8fd2-75880d156f41 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:17.086134 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 628/1272] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 15:16:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:16:17.092713 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd9c4b-b0f3-4c0b-a5a5-1e7dd6121b7b None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:17.184783 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:17.193222 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd9c4b-b0f3-4c0b-a5a5-1e7dd6121b7b None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:17.226023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1633e3b-c241-4d84-9a98-b6df36f08ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:17.226683 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1633e3b-c241-4d84-9a98-b6df36f08ab9 None admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:17.227037 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1633e3b-c241-4d84-9a98-b6df36f08ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:17.227326 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1633e3b-c241-4d84-9a98-b6df36f08ab9 None admin] PATH_INFO: `/v3/projects/06f27ab8886a4c0d81c4d4e3fa100a83/users/107a7c18a2c6440b86e1b7c3e39cc724/roles/7c88c815db854954b516ab97cd471647` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:17.229287 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1633e3b-c241-4d84-9a98-b6df36f08ab9 None admin] RBAC: Authorizing `identity:create_grant(project_id=06f27ab8886a4c0d81c4d4e3fa100a83, user_id=107a7c18a2c6440b86e1b7c3e39cc724, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:17.230787 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1633e3b-c241-4d84-9a98-b6df36f08ab9 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:17.307011 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 645/1273] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 5 15:16:17 2022] PUT /identity/v3/projects/06f27ab8886a4c0d81c4d4e3fa100a83/users/107a7c18a2c6440b86e1b7c3e39cc724/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:16:19.371496 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799cc80f-1c05-49e1-a3f3-fa879865b5d7 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:19.372600 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799cc80f-1c05-49e1-a3f3-fa879865b5d7 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:19.372600 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799cc80f-1c05-49e1-a3f3-fa879865b5d7 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:19.373980 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 629/1274] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:16:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:16:19.379217 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f134b31-cba8-466b-bf46-ec60b805617e None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:19.379631 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f134b31-cba8-466b-bf46-ec60b805617e None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:19.379773 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f134b31-cba8-466b-bf46-ec60b805617e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:19.397653 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-5f134b31-cba8-466b-bf46-ec60b805617e None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:16:19.398843 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-5f134b31-cba8-466b-bf46-ec60b805617e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:16:19.465038 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 646/1275] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:16:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:16:19.488621 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799cc80f-1c05-49e1-a3f3-fa879865b5d7 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:19.587147 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799cc80f-1c05-49e1-a3f3-fa879865b5d7 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:19.604169 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:19.621055 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80250a1f-49e9-45da-bbb1-478d8fb0fd24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:19.621705 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80250a1f-49e9-45da-bbb1-478d8fb0fd24 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:19.621986 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80250a1f-49e9-45da-bbb1-478d8fb0fd24 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:19.622260 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80250a1f-49e9-45da-bbb1-478d8fb0fd24 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:19.632023 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80250a1f-49e9-45da-bbb1-478d8fb0fd24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 15:16:19.636960 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-80250a1f-49e9-45da-bbb1-478d8fb0fd24 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:16:19.636960 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:16:19.636960 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:16:19.636960 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:16:19.636960 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:16:19.636960 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:16:19.636960 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:16:19.636960 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:16:19.636960 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:16:19.636960 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:16:19.636960 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:16:19.636960 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:16:19.636960 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 15:16:19.636960 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 15:16:19.636960 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 15:16:19.636960 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 15:16:19.638039 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 15:16:19.638039 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 15:16:19.638039 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 15:16:19.638039 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 15:16:19.638039 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:16:19.638039 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:16:19.638039 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:16:19.638039 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:16:19.638039 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:16:19.638039 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:16:19.638039 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:16:19.638039 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:16:19.638039 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:16:19.638039 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:16:19.638039 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:16:19.638039 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:16:19.639069 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:16:19.639069 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:16:19.639069 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:16:19.639069 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:16:19.639069 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 15:16:19.639069 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 15:16:19.639069 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 15:16:19.639069 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 15:16:19.639069 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 15:16:19.639069 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 15:16:19.639069 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 15:16:19.639069 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:16:19.639069 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 630/1276] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 5 15:16:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:16:19.644131 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f134b31-cba8-466b-bf46-ec60b805617e None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:19.723027 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:19.741110 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f134b31-cba8-466b-bf46-ec60b805617e None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:19.774975 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57cb4c3-b670-477f-9470-7294322b3e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:19.775810 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57cb4c3-b670-477f-9470-7294322b3e06 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:19.775909 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57cb4c3-b670-477f-9470-7294322b3e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:19.776168 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57cb4c3-b670-477f-9470-7294322b3e06 None admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:19.776881 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57cb4c3-b670-477f-9470-7294322b3e06 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:16:19.777190 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57cb4c3-b670-477f-9470-7294322b3e06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:19.779124 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57cb4c3-b670-477f-9470-7294322b3e06 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:19.787171 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 647/1277] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 5 15:16:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:16:19.792975 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80250a1f-49e9-45da-bbb1-478d8fb0fd24 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:19.894257 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80250a1f-49e9-45da-bbb1-478d8fb0fd24 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:19.927810 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19001860-34ad-4dab-bf83-012f529d84f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:19.928481 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19001860-34ad-4dab-bf83-012f529d84f6 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:19.931072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19001860-34ad-4dab-bf83-012f529d84f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:19.931072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19001860-34ad-4dab-bf83-012f529d84f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:19.938793 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19001860-34ad-4dab-bf83-012f529d84f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:19.940265 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19001860-34ad-4dab-bf83-012f529d84f6 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:19.955086 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-19001860-34ad-4dab-bf83-012f529d84f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:16:19.955086 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:16:19.955086 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:16:19.955086 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:16:19.955086 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:16:19.955086 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:16:19.955086 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:16:19.955086 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:16:19.955086 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:16:19.955086 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:16:19.955086 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:16:19.955086 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:16:19.955086 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:16:19.955086 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:16:19.955086 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:16:19.956229 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:16:19.956229 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:16:19.956229 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:16:19.956229 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:16:19.956229 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:16:19.956229 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:16:19.956229 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:16:19.956229 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:16:19.956229 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:16:19.956229 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:16:19.956229 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:16:19.956229 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:16:19.956229 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:16:19.956229 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:16:19.956229 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:16:19.956229 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:16:19.956229 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:16:19.957399 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:16:19.957399 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:16:19.957399 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:16:19.957399 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:16:19.957399 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 631/1278] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:16:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:16:19.962136 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57cb4c3-b670-477f-9470-7294322b3e06 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:20.060313 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57cb4c3-b670-477f-9470-7294322b3e06 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:20.093314 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4641dd9-be07-41f2-ad00-669b15429a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:20.093925 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4641dd9-be07-41f2-ad00-669b15429a79 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:20.094211 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4641dd9-be07-41f2-ad00-669b15429a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:20.094488 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4641dd9-be07-41f2-ad00-669b15429a79 None admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:20.095179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4641dd9-be07-41f2-ad00-669b15429a79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:16:20.095493 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4641dd9-be07-41f2-ad00-669b15429a79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:20.096890 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4641dd9-be07-41f2-ad00-669b15429a79 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:20.108369 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 648/1279] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:16:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:16:20.114039 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19001860-34ad-4dab-bf83-012f529d84f6 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:20.144785 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:20.213157 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19001860-34ad-4dab-bf83-012f529d84f6 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:20.246531 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b27057-ad31-4eb4-9bef-2897253fd029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:20.247248 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b27057-ad31-4eb4-9bef-2897253fd029 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:20.247539 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b27057-ad31-4eb4-9bef-2897253fd029 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:20.247820 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b27057-ad31-4eb4-9bef-2897253fd029 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:20.255157 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b27057-ad31-4eb4-9bef-2897253fd029 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:20.256636 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b27057-ad31-4eb4-9bef-2897253fd029 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:20.268468 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-a7b27057-ad31-4eb4-9bef-2897253fd029 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 05 15:16:20.268468 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:16:20.268468 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:16:20.268468 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:16:20.268468 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:16:20.268468 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:16:20.268468 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:16:20.268468 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:16:20.268468 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:16:20.268468 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:16:20.268468 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:16:20.268468 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:16:20.268468 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 15:16:20.268468 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 15:16:20.268468 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 15:16:20.268468 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 15:16:20.269664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:16:20.269664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:16:20.269664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 15:16:20.269664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:16:20.269664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 15:16:20.269664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 15:16:20.269664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:16:20.269664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:16:20.269664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:16:20.269664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:16:20.269664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:16:20.269664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:16:20.269664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:16:20.269664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:16:20.269664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:16:20.269664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:16:20.269664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:16:20.269664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:16:20.271021 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:16:20.271021 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:16:20.271021 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 15:16:20.271021 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 15:16:20.271021 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 15:16:20.271021 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 15:16:20.271021 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 15:16:20.271021 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 15:16:20.271021 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 05 15:16:20.271021 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:16:20.271021 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 632/1280] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:16:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:16:20.275535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4641dd9-be07-41f2-ad00-669b15429a79 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:20.279957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:20.375255 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4641dd9-be07-41f2-ad00-669b15429a79 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:20.409353 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3016ee80-d69c-4423-b33f-b678f033cf43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:20.410000 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016ee80-d69c-4423-b33f-b678f033cf43 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:20.410273 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016ee80-d69c-4423-b33f-b678f033cf43 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:20.410551 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016ee80-d69c-4423-b33f-b678f033cf43 None admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:20.411350 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3016ee80-d69c-4423-b33f-b678f033cf43 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:16:20.411664 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3016ee80-d69c-4423-b33f-b678f033cf43 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:20.413173 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3016ee80-d69c-4423-b33f-b678f033cf43 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:20.429752 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 649/1281] 10.222.0.40 () {56 vars in 1143 bytes} [Fri Aug 5 15:16:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:16:20.436383 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b27057-ad31-4eb4-9bef-2897253fd029 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:20.526664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:20.536123 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b27057-ad31-4eb4-9bef-2897253fd029 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:20.568402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0119852a-d5ab-40b4-87dd-88bbd42e97e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:20.569008 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119852a-d5ab-40b4-87dd-88bbd42e97e6 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:20.569289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119852a-d5ab-40b4-87dd-88bbd42e97e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:20.569566 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119852a-d5ab-40b4-87dd-88bbd42e97e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:20.634232 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0119852a-d5ab-40b4-87dd-88bbd42e97e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:20.635606 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0119852a-d5ab-40b4-87dd-88bbd42e97e6 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:20.734216 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 633/1282] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 5 15:16:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 15:16:20.740176 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3016ee80-d69c-4423-b33f-b678f033cf43 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:20.808339 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:20.809857 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:20.842422 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3016ee80-d69c-4423-b33f-b678f033cf43 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:20.879957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788165da-cd05-4b8f-a1d2-632f103f2cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:20.880634 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788165da-cd05-4b8f-a1d2-632f103f2cba None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:20.880920 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788165da-cd05-4b8f-a1d2-632f103f2cba None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:20.881198 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788165da-cd05-4b8f-a1d2-632f103f2cba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:20.896464 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788165da-cd05-4b8f-a1d2-632f103f2cba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:20.896464 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788165da-cd05-4b8f-a1d2-632f103f2cba None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:20.910974 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-788165da-cd05-4b8f-a1d2-632f103f2cba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:16:20.910974 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:16:20.910974 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:16:20.910974 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:16:20.910974 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:16:20.910974 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:16:20.910974 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:16:20.910974 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:16:20.910974 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:16:20.910974 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:16:20.910974 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:16:20.910974 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:16:20.910974 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 15:16:20.910974 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 15:16:20.910974 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 15:16:20.914326 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 15:16:20.914326 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:16:20.914326 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:16:20.914326 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:16:20.914326 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:16:20.914326 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:16:20.914326 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:16:20.914326 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:16:20.914326 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:16:20.914326 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:16:20.914326 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:16:20.914326 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:16:20.914326 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:16:20.914326 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:16:20.914326 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:16:20.914326 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:16:20.914326 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:16:20.915482 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 15:16:20.915482 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 15:16:20.915482 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 15:16:20.915482 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:16:20.915482 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 650/1283] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:16:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:16:20.918089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0119852a-d5ab-40b4-87dd-88bbd42e97e6 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:21.018777 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0119852a-d5ab-40b4-87dd-88bbd42e97e6 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:21.057120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964fe8ff-99a7-4e60-a9a9-e2680cd4eb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:21.057735 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964fe8ff-99a7-4e60-a9a9-e2680cd4eb27 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:21.058029 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964fe8ff-99a7-4e60-a9a9-e2680cd4eb27 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:21.058307 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964fe8ff-99a7-4e60-a9a9-e2680cd4eb27 None admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:21.059098 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964fe8ff-99a7-4e60-a9a9-e2680cd4eb27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:16:21.059927 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964fe8ff-99a7-4e60-a9a9-e2680cd4eb27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:21.061335 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964fe8ff-99a7-4e60-a9a9-e2680cd4eb27 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:21.072606 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 634/1284] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 5 15:16:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:16:21.078585 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788165da-cd05-4b8f-a1d2-632f103f2cba None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:21.189487 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788165da-cd05-4b8f-a1d2-632f103f2cba None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:21.204611 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:21.226167 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9367748-88a4-4951-838e-a01fd6631331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:21.226831 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9367748-88a4-4951-838e-a01fd6631331 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:21.227117 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9367748-88a4-4951-838e-a01fd6631331 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:21.227396 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9367748-88a4-4951-838e-a01fd6631331 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:21.235102 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9367748-88a4-4951-838e-a01fd6631331 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:21.236665 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9367748-88a4-4951-838e-a01fd6631331 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:21.251336 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application [None req-c9367748-88a4-4951-838e-a01fd6631331 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:16:21.251336 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:16:21.251336 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:16:21.251336 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:16:21.251336 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:16:21.251336 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:16:21.251336 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:16:21.251336 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:16:21.251336 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:16:21.251336 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:16:21.251336 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:16:21.251336 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:16:21.251336 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 15:16:21.251336 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 15:16:21.251336 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 15:16:21.252569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 15:16:21.252569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:16:21.252569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:16:21.252569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:16:21.252569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:16:21.252569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:16:21.252569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:16:21.252569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:16:21.252569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application with Lock( Aug 05 15:16:21.252569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:16:21.252569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:16:21.252569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:16:21.252569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:16:21.252569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:16:21.252569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:16:21.252569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:16:21.252569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:16:21.253609 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 15:16:21.253609 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 15:16:21.253609 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 15:16:21.253609 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 15:16:21.253609 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 15:16:21.253609 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 15:16:21.253609 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 15:16:21.253609 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: ERROR keystone.server.flask.application Aug 05 15:16:21.253609 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 651/1285] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 5 15:16:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:16:21.258909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964fe8ff-99a7-4e60-a9a9-e2680cd4eb27 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:21.321619 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:21.383198 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964fe8ff-99a7-4e60-a9a9-e2680cd4eb27 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:21.422199 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6ae5f-e6af-4a4f-bb17-037654dae371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:21.422745 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6ae5f-e6af-4a4f-bb17-037654dae371 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:21.423028 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6ae5f-e6af-4a4f-bb17-037654dae371 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:21.423307 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6ae5f-e6af-4a4f-bb17-037654dae371 None admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:21.424096 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e6ae5f-e6af-4a4f-bb17-037654dae371 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:16:21.424403 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e6ae5f-e6af-4a4f-bb17-037654dae371 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:21.425922 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e6ae5f-e6af-4a4f-bb17-037654dae371 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:21.439524 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 635/1286] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 5 15:16:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:16:21.445861 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9367748-88a4-4951-838e-a01fd6631331 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:21.548721 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9367748-88a4-4951-838e-a01fd6631331 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:21.583990 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952bd431-1fd4-4e11-b7db-b3fdbf9a800d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:21.584653 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952bd431-1fd4-4e11-b7db-b3fdbf9a800d None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:21.584936 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952bd431-1fd4-4e11-b7db-b3fdbf9a800d None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:21.585212 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952bd431-1fd4-4e11-b7db-b3fdbf9a800d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:21.585996 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952bd431-1fd4-4e11-b7db-b3fdbf9a800d None admin] RBAC: Adding query filter params (role.id=7c88c815db854954b516ab97cd471647, scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83, user.id=107a7c18a2c6440b86e1b7c3e39cc724) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:16:21.586311 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952bd431-1fd4-4e11-b7db-b3fdbf9a800d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:21.587751 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952bd431-1fd4-4e11-b7db-b3fdbf9a800d None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:21.593711 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 652/1287] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 5 15:16:21 2022] GET /identity/v3/role_assignments?user.id=107a7c18a2c6440b86e1b7c3e39cc724&scope.project.id=06f27ab8886a4c0d81c4d4e3fa100a83&role.id=7c88c815db854954b516ab97cd471647 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:16:23.618492 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e9149-cb9e-4dd2-8369-770fd0d1bbc7 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:23.619066 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e9149-cb9e-4dd2-8369-770fd0d1bbc7 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:23.619066 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e9149-cb9e-4dd2-8369-770fd0d1bbc7 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:23.620477 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 636/1288] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:16:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:16:23.625605 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a5ad80-0c9e-4567-96cf-db9f3350ef57 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:23.626008 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a5ad80-0c9e-4567-96cf-db9f3350ef57 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:23.626163 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a5ad80-0c9e-4567-96cf-db9f3350ef57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:23.643891 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-63a5ad80-0c9e-4567-96cf-db9f3350ef57 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:16:23.645132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-63a5ad80-0c9e-4567-96cf-db9f3350ef57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:16:23.674813 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:23.710838 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 653/1289] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:16:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 15:16:23.733265 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e9149-cb9e-4dd2-8369-770fd0d1bbc7 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:23.832387 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e9149-cb9e-4dd2-8369-770fd0d1bbc7 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:23.843260 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:23.865049 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d2046-aff1-42e2-b7d5-5ce08716597f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:23.865675 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d2046-aff1-42e2-b7d5-5ce08716597f None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:23.865958 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d2046-aff1-42e2-b7d5-5ce08716597f None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:23.866236 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d2046-aff1-42e2-b7d5-5ce08716597f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:23.866859 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d2046-aff1-42e2-b7d5-5ce08716597f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:23.868490 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d2046-aff1-42e2-b7d5-5ce08716597f None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:23.878463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-d00d2046-aff1-42e2-b7d5-5ce08716597f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 15:16:23.878463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:16:23.878463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 15:16:23.878463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 15:16:23.878463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 15:16:23.878463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 15:16:23.878463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 15:16:23.878463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:16:23.878463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 15:16:23.878463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:16:23.878463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 15:16:23.878463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:16:23.878463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:16:23.878463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:16:23.878463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:16:23.878463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:16:23.878463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:16:23.879746 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:16:23.879746 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:16:23.879746 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:16:23.879746 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:16:23.879746 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:16:23.879746 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:16:23.879746 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 15:16:23.879746 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 15:16:23.879746 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 15:16:23.879746 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 15:16:23.879746 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:16:23.879746 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:16:23.879746 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:16:23.879746 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:16:23.879746 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:16:23.879746 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:16:23.880857 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:16:23.880857 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:16:23.880857 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:16:23.880857 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:16:23.880857 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:16:23.880857 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:16:23.880857 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:16:23.880857 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:16:23.880857 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:16:23.880857 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:16:23.880857 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 15:16:23.880857 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:16:23.880857 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 15:16:23.880857 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:16:23.880857 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 637/1290] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Aug 5 15:16:23 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:16:23.884152 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a5ad80-0c9e-4567-96cf-db9f3350ef57 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:23.981294 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a5ad80-0c9e-4567-96cf-db9f3350ef57 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:24.014800 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc0d6f6-56ee-424b-baee-1f46a07363b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:24.015416 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc0d6f6-56ee-424b-baee-1f46a07363b9 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:24.015705 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc0d6f6-56ee-424b-baee-1f46a07363b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:24.015984 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc0d6f6-56ee-424b-baee-1f46a07363b9 None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:24.016704 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc0d6f6-56ee-424b-baee-1f46a07363b9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:16:24.024194 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc0d6f6-56ee-424b-baee-1f46a07363b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:24.024194 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc0d6f6-56ee-424b-baee-1f46a07363b9 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:24.024194 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 654/1291] 10.222.0.40 () {56 vars in 1116 bytes} [Fri Aug 5 15:16:23 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:16:24.029452 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d2046-aff1-42e2-b7d5-5ce08716597f None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:24.127444 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d2046-aff1-42e2-b7d5-5ce08716597f None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:24.160472 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eef38b-6148-4112-a78e-9eec7f49f74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:24.161013 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eef38b-6148-4112-a78e-9eec7f49f74a None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:24.161295 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eef38b-6148-4112-a78e-9eec7f49f74a None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:24.161569 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eef38b-6148-4112-a78e-9eec7f49f74a None admin] PATH_INFO: `/v3/services` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:24.162228 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eef38b-6148-4112-a78e-9eec7f49f74a None admin] RBAC: Adding query filter params (type=placement) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:16:24.162622 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eef38b-6148-4112-a78e-9eec7f49f74a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:24.163972 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eef38b-6148-4112-a78e-9eec7f49f74a None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:24.171749 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 638/1292] 10.222.0.40 () {56 vars in 1116 bytes} [Fri Aug 5 15:16:24 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:16:26.147993 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb777e-6bcd-435b-8089-9eaf65bd7061 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:26.148559 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb777e-6bcd-435b-8089-9eaf65bd7061 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:26.148680 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb777e-6bcd-435b-8089-9eaf65bd7061 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:26.149947 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 655/1293] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:16:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:16:26.155261 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b250848-a3b8-4e9a-b290-26e74e8c77de None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:26.155659 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b250848-a3b8-4e9a-b290-26e74e8c77de None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:26.155819 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b250848-a3b8-4e9a-b290-26e74e8c77de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:26.174588 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-8b250848-a3b8-4e9a-b290-26e74e8c77de None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:16:26.176476 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-8b250848-a3b8-4e9a-b290-26e74e8c77de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:16:26.205561 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:26.241298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 639/1294] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:16:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:16:26.264378 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bb777e-6bcd-435b-8089-9eaf65bd7061 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:26.295677 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:26.364901 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bb777e-6bcd-435b-8089-9eaf65bd7061 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:26.397961 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c489364-9e7e-4e1f-876c-f7964c052ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:26.398635 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c489364-9e7e-4e1f-876c-f7964c052ceb None admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:26.398863 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c489364-9e7e-4e1f-876c-f7964c052ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:26.399142 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c489364-9e7e-4e1f-876c-f7964c052ceb None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:26.399835 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c489364-9e7e-4e1f-876c-f7964c052ceb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:26.401244 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c489364-9e7e-4e1f-876c-f7964c052ceb None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:26.496228 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 656/1295] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 5 15:16:26 2022] POST /identity/v3/services => generated 246 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:16:28.488667 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab19f97-ccdd-4012-8dcd-51640369accf None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:28.489183 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab19f97-ccdd-4012-8dcd-51640369accf None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:28.489183 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab19f97-ccdd-4012-8dcd-51640369accf None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:28.490557 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 640/1296] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:16:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:16:28.495951 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f374794-5933-4e9a-a075-f4199bf9786e None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:28.496382 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f374794-5933-4e9a-a075-f4199bf9786e None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:28.496540 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f374794-5933-4e9a-a075-f4199bf9786e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:28.514953 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-2f374794-5933-4e9a-a075-f4199bf9786e None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:16:28.516566 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-2f374794-5933-4e9a-a075-f4199bf9786e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:16:28.588679 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 657/1297] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:16:28 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 15:16:28.611326 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab19f97-ccdd-4012-8dcd-51640369accf None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:28.716563 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab19f97-ccdd-4012-8dcd-51640369accf None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:28.753309 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92482ae8-fd2f-4b4c-b42a-a7ebae7c1d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:28.753897 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92482ae8-fd2f-4b4c-b42a-a7ebae7c1d3a None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:28.754183 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92482ae8-fd2f-4b4c-b42a-a7ebae7c1d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:28.754461 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92482ae8-fd2f-4b4c-b42a-a7ebae7c1d3a None admin] PATH_INFO: `/v3/services/placement` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:28.755085 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92482ae8-fd2f-4b4c-b42a-a7ebae7c1d3a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:28.756492 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92482ae8-fd2f-4b4c-b42a-a7ebae7c1d3a None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:28.765902 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-92482ae8-fd2f-4b4c-b42a-a7ebae7c1d3a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 15:16:28.765902 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:16:28.765902 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 15:16:28.765902 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 15:16:28.765902 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 15:16:28.765902 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 15:16:28.765902 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 15:16:28.765902 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:16:28.765902 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 15:16:28.765902 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:16:28.765902 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 15:16:28.765902 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:16:28.765902 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:16:28.765902 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:16:28.765902 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:16:28.765902 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:16:28.765902 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:16:28.769297 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:16:28.769297 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:16:28.769297 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:16:28.769297 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:16:28.769297 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:16:28.769297 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:16:28.769297 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 15:16:28.769297 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 15:16:28.769297 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 15:16:28.769297 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 15:16:28.769297 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:16:28.769297 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:16:28.769297 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:16:28.769297 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:16:28.769297 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:16:28.769297 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:16:28.770402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:16:28.770402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:16:28.770402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:16:28.770402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:16:28.770402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:16:28.770402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:16:28.770402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:16:28.770402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:16:28.770402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:16:28.770402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:16:28.770402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 15:16:28.770402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:16:28.770402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 15:16:28.770402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:16:28.770402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 641/1298] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Aug 5 15:16:28 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:16:28.773349 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f374794-5933-4e9a-a075-f4199bf9786e None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:28.791904 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:28.875425 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f374794-5933-4e9a-a075-f4199bf9786e None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:28.910391 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2f7fb4-bc6b-4956-985b-96c5140c3685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:28.911008 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f7fb4-bc6b-4956-985b-96c5140c3685 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:28.911307 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f7fb4-bc6b-4956-985b-96c5140c3685 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:28.911646 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f7fb4-bc6b-4956-985b-96c5140c3685 None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:28.912350 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2f7fb4-bc6b-4956-985b-96c5140c3685 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:16:28.912661 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2f7fb4-bc6b-4956-985b-96c5140c3685 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:28.914026 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2f7fb4-bc6b-4956-985b-96c5140c3685 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:28.919211 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 658/1299] 10.222.0.40 () {56 vars in 1116 bytes} [Fri Aug 5 15:16:28 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:16:28.925291 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92482ae8-fd2f-4b4c-b42a-a7ebae7c1d3a None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:28.958385 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:29.033783 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92482ae8-fd2f-4b4c-b42a-a7ebae7c1d3a None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:29.067384 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee86d2f3-07d6-47f7-b21e-bdc57197d17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:29.067989 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee86d2f3-07d6-47f7-b21e-bdc57197d17f None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:29.068299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee86d2f3-07d6-47f7-b21e-bdc57197d17f None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:29.068575 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee86d2f3-07d6-47f7-b21e-bdc57197d17f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:29.069326 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee86d2f3-07d6-47f7-b21e-bdc57197d17f None admin] RBAC: Adding query filter params (interface=public, service_id=9fac994fd3bb452083ee07634caa0d2d, region_id=RegionOne) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:16:29.069639 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee86d2f3-07d6-47f7-b21e-bdc57197d17f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:29.071005 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee86d2f3-07d6-47f7-b21e-bdc57197d17f None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:29.075872 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 642/1300] 10.222.0.40 () {56 vars in 1251 bytes} [Fri Aug 5 15:16:28 2022] GET /identity/v3/endpoints?service_id=9fac994fd3bb452083ee07634caa0d2d&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:16:29.081713 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2f7fb4-bc6b-4956-985b-96c5140c3685 None admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:29.161986 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:29.180327 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2f7fb4-bc6b-4956-985b-96c5140c3685 None admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:29.214075 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932d524-6046-4f18-b7cd-d4f84c675846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:29.214689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932d524-6046-4f18-b7cd-d4f84c675846 None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:29.214972 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932d524-6046-4f18-b7cd-d4f84c675846 None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:29.215256 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932d524-6046-4f18-b7cd-d4f84c675846 None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:29.216001 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7932d524-6046-4f18-b7cd-d4f84c675846 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:29.217421 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7932d524-6046-4f18-b7cd-d4f84c675846 None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:29.224117 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 659/1301] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Aug 5 15:16:29 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:16:31.194362 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2401a-f3a2-4ad4-9214-3a5cffe536d2 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:31.194842 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2401a-f3a2-4ad4-9214-3a5cffe536d2 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:31.194933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2401a-f3a2-4ad4-9214-3a5cffe536d2 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:31.196298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 643/1302] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:16:31.202054 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f235e-a4d7-497e-9385-53742c19f1a9 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:31.204971 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f235e-a4d7-497e-9385-53742c19f1a9 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:31.204971 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f235e-a4d7-497e-9385-53742c19f1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:31.220361 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-892f235e-a4d7-497e-9385-53742c19f1a9 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:16:31.221534 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-892f235e-a4d7-497e-9385-53742c19f1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:16:31.287249 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 660/1303] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:16:31 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 15:16:31.310795 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2401a-f3a2-4ad4-9214-3a5cffe536d2 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:31.404283 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:31.410939 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2401a-f3a2-4ad4-9214-3a5cffe536d2 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:31.445544 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeaa332-b008-4d83-b356-ddb47b087551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:31.446194 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeaa332-b008-4d83-b356-ddb47b087551 None admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:31.446480 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeaa332-b008-4d83-b356-ddb47b087551 None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:31.446759 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeaa332-b008-4d83-b356-ddb47b087551 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:31.447382 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeaa332-b008-4d83-b356-ddb47b087551 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:31.448802 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeaa332-b008-4d83-b356-ddb47b087551 None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:31.458091 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application [None req-bbeaa332-b008-4d83-b356-ddb47b087551 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 15:16:31.458091 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:16:31.458091 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 15:16:31.458091 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 15:16:31.458091 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 15:16:31.458091 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 15:16:31.458091 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 15:16:31.458091 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:16:31.458091 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 15:16:31.458091 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:16:31.458091 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 15:16:31.458091 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:16:31.458091 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 15:16:31.458091 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 15:16:31.458091 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 15:16:31.458091 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 15:16:31.458091 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 15:16:31.459299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 15:16:31.459299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 15:16:31.459299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 15:16:31.459299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 15:16:31.459299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 15:16:31.459299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 15:16:31.459299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 15:16:31.459299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 15:16:31.459299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 15:16:31.459299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 15:16:31.459299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 15:16:31.459299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 15:16:31.459299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 15:16:31.459299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 15:16:31.459299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 15:16:31.459299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 15:16:31.460276 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 15:16:31.460276 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application with Lock( Aug 05 15:16:31.460276 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 15:16:31.460276 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self._enter() Aug 05 15:16:31.460276 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 15:16:31.460276 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 15:16:31.460276 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 15:16:31.460276 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application return self.creator() Aug 05 15:16:31.460276 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 15:16:31.460276 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application created_value = creator( Aug 05 15:16:31.460276 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 15:16:31.460276 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 15:16:31.460276 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 15:16:31.460276 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: ERROR keystone.server.flask.application Aug 05 15:16:31.460276 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 644/1304] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Aug 5 15:16:31 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 15:16:31.465784 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f235e-a4d7-497e-9385-53742c19f1a9 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:31.542163 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:31.565640 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f235e-a4d7-497e-9385-53742c19f1a9 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:31.598249 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a1b084-70e7-4198-8970-b3ccca0d153b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:31.603330 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1b084-70e7-4198-8970-b3ccca0d153b None admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:31.603330 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1b084-70e7-4198-8970-b3ccca0d153b None admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:31.603330 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1b084-70e7-4198-8970-b3ccca0d153b None admin] PATH_INFO: `/v3/services` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:31.603330 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a1b084-70e7-4198-8970-b3ccca0d153b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:16:31.603330 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a1b084-70e7-4198-8970-b3ccca0d153b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:31.603330 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a1b084-70e7-4198-8970-b3ccca0d153b None admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:31.607327 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 661/1305] 10.222.0.40 () {56 vars in 1116 bytes} [Fri Aug 5 15:16:31 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:16:31.613354 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeaa332-b008-4d83-b356-ddb47b087551 None admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:16:31.712654 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeaa332-b008-4d83-b356-ddb47b087551 None admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:16:31.746080 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e5c495-332e-4838-bdc5-7aea190b744e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:16:31.752537 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5c495-332e-4838-bdc5-7aea190b744e None admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:16:31.752537 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5c495-332e-4838-bdc5-7aea190b744e None admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:16:31.752537 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5c495-332e-4838-bdc5-7aea190b744e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:16:31.752537 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e5c495-332e-4838-bdc5-7aea190b744e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:16:31.752537 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e5c495-332e-4838-bdc5-7aea190b744e None admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:16:31.831865 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 645/1306] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:16:31 2022] POST /identity/v3/endpoints => generated 331 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:16:41.628011 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=30 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:16:41.827297 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:22:30.466556 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1941238-d706-49bb-b968-d7dfce73c30e None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:22:30.467577 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1941238-d706-49bb-b968-d7dfce73c30e None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:22:30.468110 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1941238-d706-49bb-b968-d7dfce73c30e None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:22:30.474853 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb31663-a298-41ed-9eaa-1591bbd27b01 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:22:30.485307 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb31663-a298-41ed-9eaa-1591bbd27b01 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:22:30.485307 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb31663-a298-41ed-9eaa-1591bbd27b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:22:30.486195 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 662/1307] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:22:30 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:22:30.638897 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-cbb31663-a298-41ed-9eaa-1591bbd27b01 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:22:30.640609 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-cbb31663-a298-41ed-9eaa-1591bbd27b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:22:30.833259 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 646/1308] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:22:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:22:30.883975 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a4a53-cd17-40e3-acc5-51504bb23671 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:22:30.884670 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a4a53-cd17-40e3-acc5-51504bb23671 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:22:30.884962 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a4a53-cd17-40e3-acc5-51504bb23671 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:22:30.886525 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 663/1309] 10.222.0.40 () {54 vars in 965 bytes} [Fri Aug 5 15:22:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:22:30.892671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1bea0b-2861-4dfc-8054-e6e9031315c2 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:22:30.893173 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1bea0b-2861-4dfc-8054-e6e9031315c2 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:22:30.893757 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1bea0b-2861-4dfc-8054-e6e9031315c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:22:30.974364 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-ce1bea0b-2861-4dfc-8054-e6e9031315c2 None None] MFA Rules not processed for user `d13a1638b2ca497fb02c71615559d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:22:30.975984 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-ce1bea0b-2861-4dfc-8054-e6e9031315c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:22:30.998427 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:22:31.064717 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 647/1310] 10.222.0.40 () {58 vars in 1062 bytes} [Fri Aug 5 15:22:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:22:31.073994 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1a4a53-cd17-40e3-acc5-51504bb23671 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:22:31.075818 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-8f1a4a53-cd17-40e3-acc5-51504bb23671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:22:31.194252 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1a4a53-cd17-40e3-acc5-51504bb23671 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:22:31.273350 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0128f6b-e674-460d-9996-3f4275982ec5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:22:31.273979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0128f6b-e674-460d-9996-3f4275982ec5 service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:22:31.274262 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0128f6b-e674-460d-9996-3f4275982ec5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:22:31.274537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0128f6b-e674-460d-9996-3f4275982ec5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:22:31.276755 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-e0128f6b-e674-460d-9996-3f4275982ec5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:22:31.336707 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:22:31.402584 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0128f6b-e674-460d-9996-3f4275982ec5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:22:31.404265 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0128f6b-e674-460d-9996-3f4275982ec5 service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:22:31.425462 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 664/1311] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:22:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:22:33.731314 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5b0d5-1da9-4bbe-bb61-e11b54aec5d3 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:22:33.731859 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5b0d5-1da9-4bbe-bb61-e11b54aec5d3 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:22:33.731859 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5b0d5-1da9-4bbe-bb61-e11b54aec5d3 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:22:33.733418 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 648/1312] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:22:33.739104 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad1784-24a7-483b-bcbb-1821c6a709b8 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:22:33.739612 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad1784-24a7-483b-bcbb-1821c6a709b8 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:22:33.739704 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad1784-24a7-483b-bcbb-1821c6a709b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:22:33.823769 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-24ad1784-24a7-483b-bcbb-1821c6a709b8 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:22:33.825526 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-24ad1784-24a7-483b-bcbb-1821c6a709b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:22:33.937888 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 665/1313] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:22:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:22:33.976117 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5b0d5-1da9-4bbe-bb61-e11b54aec5d3 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:22:33.977780 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-a5f5b0d5-1da9-4bbe-bb61-e11b54aec5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:22:34.086265 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5b0d5-1da9-4bbe-bb61-e11b54aec5d3 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:22:34.169623 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:22:34.172600 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b08ce-7d4a-4f21-9ab6-9268faa50244 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:22:34.173226 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b08ce-7d4a-4f21-9ab6-9268faa50244 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:22:34.173517 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b08ce-7d4a-4f21-9ab6-9268faa50244 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:22:34.173799 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b08ce-7d4a-4f21-9ab6-9268faa50244 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:22:34.175453 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-d59b08ce-7d4a-4f21-9ab6-9268faa50244 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:22:34.286449 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b08ce-7d4a-4f21-9ab6-9268faa50244 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:22:34.287888 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b08ce-7d4a-4f21-9ab6-9268faa50244 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:22:34.309547 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 649/1314] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:22:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:22:36.868920 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca987ed-b974-4e80-8321-97a4764c6d55 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:22:36.869484 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca987ed-b974-4e80-8321-97a4764c6d55 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:22:36.869484 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca987ed-b974-4e80-8321-97a4764c6d55 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:22:36.871362 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 666/1315] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:22:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:22:36.876453 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3179b-9edd-4f1c-bad8-87ec3376f948 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:22:36.876848 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3179b-9edd-4f1c-bad8-87ec3376f948 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:22:36.877008 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3179b-9edd-4f1c-bad8-87ec3376f948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:22:36.941077 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-4aa3179b-9edd-4f1c-bad8-87ec3376f948 None None] MFA Rules not processed for user `a6cf4078f01748e98571f285e560294e`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:22:36.942319 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-4aa3179b-9edd-4f1c-bad8-87ec3376f948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:22:36.992519 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:22:37.037385 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 650/1316] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:22:36 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:22:37.077778 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca987ed-b974-4e80-8321-97a4764c6d55 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:22:37.079326 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-9ca987ed-b974-4e80-8321-97a4764c6d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:22:37.089018 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:22:37.181505 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca987ed-b974-4e80-8321-97a4764c6d55 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:22:37.256365 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b6b43-e8c1-4235-9e10-02e2c245d05e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:22:37.256996 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b6b43-e8c1-4235-9e10-02e2c245d05e service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:22:37.257450 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b6b43-e8c1-4235-9e10-02e2c245d05e service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:22:37.257732 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b6b43-e8c1-4235-9e10-02e2c245d05e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:22:37.259384 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-1d1b6b43-e8c1-4235-9e10-02e2c245d05e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:22:37.322199 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:22:37.370735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1b6b43-e8c1-4235-9e10-02e2c245d05e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:22:37.372247 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1b6b43-e8c1-4235-9e10-02e2c245d05e service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:22:37.392263 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 667/1317] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:22:37 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:22:40.365878 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186483d4-5d91-4e56-9ec8-53ea8daaf068 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:22:40.366475 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186483d4-5d91-4e56-9ec8-53ea8daaf068 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:22:40.366475 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186483d4-5d91-4e56-9ec8-53ea8daaf068 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:22:40.367831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 651/1318] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:22:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:22:40.374253 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138a475-4dc6-451a-bf9f-9ca14391d99c None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:22:40.374716 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138a475-4dc6-451a-bf9f-9ca14391d99c None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:22:40.374810 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138a475-4dc6-451a-bf9f-9ca14391d99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:22:40.439147 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-6138a475-4dc6-451a-bf9f-9ca14391d99c None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:22:40.440331 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-6138a475-4dc6-451a-bf9f-9ca14391d99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:22:40.533424 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 668/1319] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:22:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:22:40.573355 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186483d4-5d91-4e56-9ec8-53ea8daaf068 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:22:40.574884 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-186483d4-5d91-4e56-9ec8-53ea8daaf068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:22:40.675904 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186483d4-5d91-4e56-9ec8-53ea8daaf068 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:22:40.747872 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb5ebf-b75f-4487-abda-fd77746f9d5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:22:40.748452 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb5ebf-b75f-4487-abda-fd77746f9d5f service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:22:40.748743 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb5ebf-b75f-4487-abda-fd77746f9d5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:22:40.749029 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb5ebf-b75f-4487-abda-fd77746f9d5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:22:40.750670 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-f3bb5ebf-b75f-4487-abda-fd77746f9d5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:22:40.758406 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:22:40.858554 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bb5ebf-b75f-4487-abda-fd77746f9d5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:22:40.859984 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bb5ebf-b75f-4487-abda-fd77746f9d5f service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:22:40.877969 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 652/1320] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:22:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:22:43.962198 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a75a7-7a94-46b8-8079-c1e71d5ffb7e None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:22:43.962741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a75a7-7a94-46b8-8079-c1e71d5ffb7e None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:22:43.962741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a75a7-7a94-46b8-8079-c1e71d5ffb7e None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:22:43.964136 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 669/1321] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:22:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:22:43.969476 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ca878-7364-4625-8c03-15bf84a7f120 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:22:43.969919 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ca878-7364-4625-8c03-15bf84a7f120 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:22:43.970030 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ca878-7364-4625-8c03-15bf84a7f120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:22:44.044837 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-9a5ca878-7364-4625-8c03-15bf84a7f120 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:22:44.046112 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-9a5ca878-7364-4625-8c03-15bf84a7f120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:22:44.107320 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:22:44.149634 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 653/1322] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:22:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:22:44.187494 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a75a7-7a94-46b8-8079-c1e71d5ffb7e None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:22:44.189158 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-690a75a7-7a94-46b8-8079-c1e71d5ffb7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:22:44.208747 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:22:44.293348 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a75a7-7a94-46b8-8079-c1e71d5ffb7e None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:22:44.366158 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d4b72a-642f-46f0-8f9e-884886806912 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:22:44.366779 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d4b72a-642f-46f0-8f9e-884886806912 service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:22:44.367066 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d4b72a-642f-46f0-8f9e-884886806912 service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:22:44.367341 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d4b72a-642f-46f0-8f9e-884886806912 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:22:44.369002 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-73d4b72a-642f-46f0-8f9e-884886806912 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:22:44.442006 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:22:44.480185 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d4b72a-642f-46f0-8f9e-884886806912 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:22:44.481765 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d4b72a-642f-46f0-8f9e-884886806912 service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:22:44.499591 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 670/1323] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:22:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:22:46.567387 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4b45ac-31f1-4071-904e-f877bbab976c None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:22:46.567949 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4b45ac-31f1-4071-904e-f877bbab976c None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:22:46.567949 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4b45ac-31f1-4071-904e-f877bbab976c None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:22:46.569342 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 654/1324] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:22:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:22:46.575259 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099decf-4983-401d-aae4-0d7276bb177d None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:22:46.575702 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099decf-4983-401d-aae4-0d7276bb177d None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:22:46.575806 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099decf-4983-401d-aae4-0d7276bb177d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:22:46.643005 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-8099decf-4983-401d-aae4-0d7276bb177d None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:22:46.644269 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-8099decf-4983-401d-aae4-0d7276bb177d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:22:46.744964 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 671/1325] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:22:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:22:46.796882 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f3b59-e95b-4906-9814-605e060384ee None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:22:46.797326 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f3b59-e95b-4906-9814-605e060384ee None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:22:46.797651 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f3b59-e95b-4906-9814-605e060384ee None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:22:46.799096 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 655/1326] 10.222.0.40 () {54 vars in 965 bytes} [Fri Aug 5 15:22:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:22:46.805118 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732314bf-aae3-462c-8ea5-27051739c3ef None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:22:46.805415 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732314bf-aae3-462c-8ea5-27051739c3ef None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:22:46.805698 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732314bf-aae3-462c-8ea5-27051739c3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:22:46.838000 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:22:46.883394 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-732314bf-aae3-462c-8ea5-27051739c3ef None None] MFA Rules not processed for user `d13a1638b2ca497fb02c71615559d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:22:46.885273 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-732314bf-aae3-462c-8ea5-27051739c3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:22:46.966148 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 672/1327] 10.222.0.40 () {58 vars in 1062 bytes} [Fri Aug 5 15:22:46 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:22:46.975502 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f3b59-e95b-4906-9814-605e060384ee None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:22:46.977083 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-a62f3b59-e95b-4906-9814-605e060384ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:22:47.081139 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f3b59-e95b-4906-9814-605e060384ee None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:22:47.162443 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a512aa34-21df-4086-87f2-7e93a7415464 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:22:47.163063 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a512aa34-21df-4086-87f2-7e93a7415464 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:22:47.163345 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a512aa34-21df-4086-87f2-7e93a7415464 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:22:47.163653 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a512aa34-21df-4086-87f2-7e93a7415464 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:22:47.165356 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-a512aa34-21df-4086-87f2-7e93a7415464 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:22:47.183588 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:22:47.275736 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a512aa34-21df-4086-87f2-7e93a7415464 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:22:47.277225 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a512aa34-21df-4086-87f2-7e93a7415464 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:22:47.295832 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 656/1328] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:22:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:22:49.564675 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c7f06-0b08-48be-92f1-38886cb5c427 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:22:49.565179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c7f06-0b08-48be-92f1-38886cb5c427 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:22:49.565272 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c7f06-0b08-48be-92f1-38886cb5c427 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:22:49.566604 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 673/1329] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:22:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:22:49.572532 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745cadf5-1a15-467c-abc2-8abf4e6e55d5 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:22:49.572960 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745cadf5-1a15-467c-abc2-8abf4e6e55d5 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:22:49.573089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745cadf5-1a15-467c-abc2-8abf4e6e55d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:22:49.638584 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-745cadf5-1a15-467c-abc2-8abf4e6e55d5 None None] MFA Rules not processed for user `a6cf4078f01748e98571f285e560294e`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:22:49.639790 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-745cadf5-1a15-467c-abc2-8abf4e6e55d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:22:49.702265 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:22:49.754228 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 657/1330] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:22:49 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:22:49.792672 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c7f06-0b08-48be-92f1-38886cb5c427 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:22:49.794243 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-0c3c7f06-0b08-48be-92f1-38886cb5c427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:22:49.900825 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c7f06-0b08-48be-92f1-38886cb5c427 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:22:49.920376 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:22:49.973855 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97ed18-2fb7-47fc-8c5b-093c08e7e820 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:22:49.974407 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97ed18-2fb7-47fc-8c5b-093c08e7e820 service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:22:49.974693 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97ed18-2fb7-47fc-8c5b-093c08e7e820 service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:22:49.974971 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97ed18-2fb7-47fc-8c5b-093c08e7e820 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:22:49.976626 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-5d97ed18-2fb7-47fc-8c5b-093c08e7e820 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:22:50.087458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d97ed18-2fb7-47fc-8c5b-093c08e7e820 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:22:50.089007 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d97ed18-2fb7-47fc-8c5b-093c08e7e820 service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:22:50.107792 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 674/1331] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:22:49 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:22:52.713112 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa3044-1582-40e9-aa43-df040133861f None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:22:52.713653 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa3044-1582-40e9-aa43-df040133861f None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:22:52.713653 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa3044-1582-40e9-aa43-df040133861f None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:22:52.715027 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 658/1332] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:22:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:22:52.720408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe98c473-f7f3-4fd5-80f7-dc59116c4cb1 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:22:52.720815 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe98c473-f7f3-4fd5-80f7-dc59116c4cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:22:52.720964 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe98c473-f7f3-4fd5-80f7-dc59116c4cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:22:52.787748 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:22:52.792883 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-fe98c473-f7f3-4fd5-80f7-dc59116c4cb1 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:22:52.794143 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-fe98c473-f7f3-4fd5-80f7-dc59116c4cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:22:52.892432 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 675/1333] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:22:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:22:52.927931 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fa3044-1582-40e9-aa43-df040133861f None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:22:52.929533 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-b5fa3044-1582-40e9-aa43-df040133861f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:22:53.032123 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fa3044-1582-40e9-aa43-df040133861f None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:22:53.108727 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5090be-c9df-4407-bc51-0e3f3a3c884d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:22:53.109622 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5090be-c9df-4407-bc51-0e3f3a3c884d service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:22:53.109913 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5090be-c9df-4407-bc51-0e3f3a3c884d service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:22:53.110196 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5090be-c9df-4407-bc51-0e3f3a3c884d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:22:53.112188 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-7e5090be-c9df-4407-bc51-0e3f3a3c884d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:22:53.139803 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:22:53.220400 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5090be-c9df-4407-bc51-0e3f3a3c884d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:22:53.221837 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5090be-c9df-4407-bc51-0e3f3a3c884d service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:22:53.240481 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 659/1334] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:22:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:22:55.283402 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a32cf7-6b70-42c9-8398-660458e5b636 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:22:55.283987 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a32cf7-6b70-42c9-8398-660458e5b636 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:22:55.283987 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a32cf7-6b70-42c9-8398-660458e5b636 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:22:55.285345 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 676/1335] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:22:55.290574 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9a516-68cf-48c0-a56b-ad2602dbdadc None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:22:55.291002 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9a516-68cf-48c0-a56b-ad2602dbdadc None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:22:55.291160 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9a516-68cf-48c0-a56b-ad2602dbdadc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:22:55.355664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-73d9a516-68cf-48c0-a56b-ad2602dbdadc None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:22:55.356900 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-73d9a516-68cf-48c0-a56b-ad2602dbdadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:22:55.438049 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:22:55.463058 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 660/1336] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:22:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:22:55.503230 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a32cf7-6b70-42c9-8398-660458e5b636 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:22:55.504836 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-53a32cf7-6b70-42c9-8398-660458e5b636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:22:55.607547 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a32cf7-6b70-42c9-8398-660458e5b636 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:22:55.636704 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:22:55.680623 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c0b8e-4f89-421f-85b9-3dd3f0ebd53d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:22:55.681746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c0b8e-4f89-421f-85b9-3dd3f0ebd53d service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:22:55.682029 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c0b8e-4f89-421f-85b9-3dd3f0ebd53d service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:22:55.682304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c0b8e-4f89-421f-85b9-3dd3f0ebd53d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:22:55.683925 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-f07c0b8e-4f89-421f-85b9-3dd3f0ebd53d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:22:55.796903 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c0b8e-4f89-421f-85b9-3dd3f0ebd53d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:22:55.798362 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c0b8e-4f89-421f-85b9-3dd3f0ebd53d service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:22:55.818098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 677/1337] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:22:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:22:58.527834 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a6118-ea47-46f8-85c6-d51b457a0db2 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:22:58.528383 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a6118-ea47-46f8-85c6-d51b457a0db2 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:22:58.528479 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a6118-ea47-46f8-85c6-d51b457a0db2 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:22:58.529808 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 661/1338] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:22:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:22:58.535075 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250494e0-c751-4906-882d-9664b758da40 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:22:58.535513 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250494e0-c751-4906-882d-9664b758da40 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:22:58.535628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250494e0-c751-4906-882d-9664b758da40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:22:58.599505 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-250494e0-c751-4906-882d-9664b758da40 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:22:58.600742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-250494e0-c751-4906-882d-9664b758da40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:22:58.608867 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:22:58.695521 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 678/1339] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:22:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:22:58.732419 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90a6118-ea47-46f8-85c6-d51b457a0db2 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:22:58.733959 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-f90a6118-ea47-46f8-85c6-d51b457a0db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:22:58.839844 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90a6118-ea47-46f8-85c6-d51b457a0db2 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:22:58.910773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2884e00e-764a-4975-9097-f1eafc8145f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:22:58.911385 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884e00e-764a-4975-9097-f1eafc8145f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:22:58.911666 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884e00e-764a-4975-9097-f1eafc8145f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:22:58.911936 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884e00e-764a-4975-9097-f1eafc8145f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:22:58.913580 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-2884e00e-764a-4975-9097-f1eafc8145f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:22:58.948519 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:22:59.019430 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2884e00e-764a-4975-9097-f1eafc8145f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:22:59.020881 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2884e00e-764a-4975-9097-f1eafc8145f2 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:22:59.038653 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 662/1340] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:22:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:23:03.263149 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ed142-b762-42a5-9478-bacfdb8e79e4 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:03.263708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ed142-b762-42a5-9478-bacfdb8e79e4 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:03.264170 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ed142-b762-42a5-9478-bacfdb8e79e4 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:03.266175 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 679/1341] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:23:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:23:03.271323 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7104902-8a11-4b2d-9b24-e35d6544c1c8 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:03.271778 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7104902-8a11-4b2d-9b24-e35d6544c1c8 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:03.271948 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7104902-8a11-4b2d-9b24-e35d6544c1c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:03.342164 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-e7104902-8a11-4b2d-9b24-e35d6544c1c8 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:23:03.343299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-e7104902-8a11-4b2d-9b24-e35d6544c1c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:03.428841 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:03.440265 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 663/1342] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:23:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:23:03.479982 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11ed142-b762-42a5-9478-bacfdb8e79e4 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:23:03.481618 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-b11ed142-b762-42a5-9478-bacfdb8e79e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:03.589210 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11ed142-b762-42a5-9478-bacfdb8e79e4 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:23:03.625794 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:03.662836 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f9be63-20a1-4362-be9b-552433adf6b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:23:03.663451 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9be63-20a1-4362-be9b-552433adf6b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:03.663734 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9be63-20a1-4362-be9b-552433adf6b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:03.664009 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9be63-20a1-4362-be9b-552433adf6b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:03.665735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-52f9be63-20a1-4362-be9b-552433adf6b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:03.779321 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f9be63-20a1-4362-be9b-552433adf6b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:23:03.780817 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f9be63-20a1-4362-be9b-552433adf6b1 service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:23:03.798595 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 680/1343] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:23:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:23:06.768425 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894f1b8-461e-482f-813b-4f74f09eb9fa None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:06.768980 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894f1b8-461e-482f-813b-4f74f09eb9fa None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:06.768980 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894f1b8-461e-482f-813b-4f74f09eb9fa None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:06.770476 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 664/1344] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:23:06.776108 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce3ad1-67a0-48d8-b2f3-cbe6505c2d81 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:06.776555 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce3ad1-67a0-48d8-b2f3-cbe6505c2d81 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:06.776650 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce3ad1-67a0-48d8-b2f3-cbe6505c2d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:06.843642 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-4fce3ad1-67a0-48d8-b2f3-cbe6505c2d81 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:23:06.844871 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-4fce3ad1-67a0-48d8-b2f3-cbe6505c2d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:06.862625 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:06.941357 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 681/1345] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:23:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:23:06.992448 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5894f1b8-461e-482f-813b-4f74f09eb9fa None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:23:06.994583 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-5894f1b8-461e-482f-813b-4f74f09eb9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:07.099663 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5894f1b8-461e-482f-813b-4f74f09eb9fa None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:23:07.174985 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d4958-a433-4242-98f7-70ccbed67338 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:23:07.175617 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d4958-a433-4242-98f7-70ccbed67338 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:07.175905 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d4958-a433-4242-98f7-70ccbed67338 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:07.176226 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d4958-a433-4242-98f7-70ccbed67338 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:07.177875 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-3d0d4958-a433-4242-98f7-70ccbed67338 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:07.221383 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:07.288992 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0d4958-a433-4242-98f7-70ccbed67338 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:23:07.291039 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0d4958-a433-4242-98f7-70ccbed67338 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:23:07.315095 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 665/1346] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:23:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:23:11.532032 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c39cb-2262-4692-9038-49a571df262c None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:11.532677 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c39cb-2262-4692-9038-49a571df262c None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:11.532777 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c39cb-2262-4692-9038-49a571df262c None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:11.534072 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 682/1347] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:23:11.539214 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a0229-9264-4692-aace-c8ad50001a06 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:11.539598 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a0229-9264-4692-aace-c8ad50001a06 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:11.539826 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a0229-9264-4692-aace-c8ad50001a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:11.615805 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-704a0229-9264-4692-aace-c8ad50001a06 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:23:11.617300 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-704a0229-9264-4692-aace-c8ad50001a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:11.718094 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:11.721313 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 666/1348] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:23:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:23:11.761787 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c39cb-2262-4692-9038-49a571df262c None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:23:11.763365 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-109c39cb-2262-4692-9038-49a571df262c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:11.877394 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c39cb-2262-4692-9038-49a571df262c None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:23:11.928141 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:11.955748 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32510b8f-a125-4079-9d62-07e0d5c0c9b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:23:11.956556 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32510b8f-a125-4079-9d62-07e0d5c0c9b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:11.956749 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32510b8f-a125-4079-9d62-07e0d5c0c9b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:11.957025 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32510b8f-a125-4079-9d62-07e0d5c0c9b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:11.958655 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-32510b8f-a125-4079-9d62-07e0d5c0c9b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:12.073294 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32510b8f-a125-4079-9d62-07e0d5c0c9b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:23:12.074740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32510b8f-a125-4079-9d62-07e0d5c0c9b8 service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:23:12.093657 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 683/1349] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:23:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:23:16.997062 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a4a8c-65fe-4529-8045-86c13940ddbb None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:16.997499 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a4a8c-65fe-4529-8045-86c13940ddbb None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:16.997610 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a4a8c-65fe-4529-8045-86c13940ddbb None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:16.998986 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 667/1350] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 5 15:23:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:23:17.004621 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70e088-71c3-440d-ab92-2af8b29cd3a3 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:17.005039 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70e088-71c3-440d-ab92-2af8b29cd3a3 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:17.005242 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70e088-71c3-440d-ab92-2af8b29cd3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:17.167385 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-de70e088-71c3-440d-ab92-2af8b29cd3a3 None None] MFA Rules not processed for user `107a7c18a2c6440b86e1b7c3e39cc724`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:23:17.168872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-de70e088-71c3-440d-ab92-2af8b29cd3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:17.189315 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:17.282492 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 684/1351] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 5 15:23:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:23:21.455633 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ccd0cd-1ecd-4fbc-9fb3-f6a518fa360a None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:21.456165 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ccd0cd-1ecd-4fbc-9fb3-f6a518fa360a None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:21.456165 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ccd0cd-1ecd-4fbc-9fb3-f6a518fa360a None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:21.457749 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 668/1352] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 5 15:23:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:23:21.463396 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c39036e-eb10-40e6-95b6-f21486d99966 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:21.464396 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c39036e-eb10-40e6-95b6-f21486d99966 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:21.464889 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c39036e-eb10-40e6-95b6-f21486d99966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:21.543598 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-2c39036e-eb10-40e6-95b6-f21486d99966 None None] MFA Rules not processed for user `107a7c18a2c6440b86e1b7c3e39cc724`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:23:21.545216 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-2c39036e-eb10-40e6-95b6-f21486d99966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:21.621954 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:21.660737 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 685/1353] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 5 15:23:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:23:23.086768 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3cb42-51e1-4562-b70c-b621800a4beb None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:23.087310 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3cb42-51e1-4562-b70c-b621800a4beb None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:23.087310 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3cb42-51e1-4562-b70c-b621800a4beb None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:23.088702 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 669/1354] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 5 15:23:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:23:23.093708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1549f3-6200-4d04-9885-e14dacc8746a None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:23.094095 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1549f3-6200-4d04-9885-e14dacc8746a None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:23.094274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1549f3-6200-4d04-9885-e14dacc8746a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:23.169332 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-6c1549f3-6200-4d04-9885-e14dacc8746a None None] MFA Rules not processed for user `107a7c18a2c6440b86e1b7c3e39cc724`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:23:23.170567 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-6c1549f3-6200-4d04-9885-e14dacc8746a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:23.285820 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 686/1355] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 5 15:23:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:23:26.599791 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a4859-a267-48db-a1a0-f5d4585f6e9c None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:26.600833 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a4859-a267-48db-a1a0-f5d4585f6e9c None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:26.601155 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a4859-a267-48db-a1a0-f5d4585f6e9c None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:26.602726 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 670/1356] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:23:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:23:26.608298 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed610333-5636-4842-b1f3-4ce21fd8e257 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:26.608774 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed610333-5636-4842-b1f3-4ce21fd8e257 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:26.608890 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed610333-5636-4842-b1f3-4ce21fd8e257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:26.658508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:26.690803 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-ed610333-5636-4842-b1f3-4ce21fd8e257 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:23:26.692015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-ed610333-5636-4842-b1f3-4ce21fd8e257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:26.800510 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 687/1357] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:23:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:23:26.857941 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c21b0-064e-46db-93fc-bda53cb91e04 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:26.858571 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c21b0-064e-46db-93fc-bda53cb91e04 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:26.859245 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c21b0-064e-46db-93fc-bda53cb91e04 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:26.861656 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 671/1358] 10.222.0.40 () {54 vars in 962 bytes} [Fri Aug 5 15:23:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:23:26.869920 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70679d6a-f804-4496-a53c-b7aa23b8a19f None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:26.873828 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70679d6a-f804-4496-a53c-b7aa23b8a19f None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:26.873828 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70679d6a-f804-4496-a53c-b7aa23b8a19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:26.956812 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-70679d6a-f804-4496-a53c-b7aa23b8a19f None None] MFA Rules not processed for user `bc0497dab1f44bb7b8196d39bf01d770`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:23:26.958005 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-70679d6a-f804-4496-a53c-b7aa23b8a19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:27.010440 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:27.081974 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 688/1359] 10.222.0.40 () {58 vars in 1059 bytes} [Fri Aug 5 15:23:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:23:27.090611 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c21b0-064e-46db-93fc-bda53cb91e04 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:23:27.092159 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-2e1c21b0-064e-46db-93fc-bda53cb91e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:27.242642 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c21b0-064e-46db-93fc-bda53cb91e04 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:23:27.338121 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:27.391382 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4747453a-d4c4-4a88-91b9-83ac26e25e9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:23:27.392493 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747453a-d4c4-4a88-91b9-83ac26e25e9c service nova] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:27.393120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747453a-d4c4-4a88-91b9-83ac26e25e9c service nova] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:27.393603 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747453a-d4c4-4a88-91b9-83ac26e25e9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:27.396481 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-4747453a-d4c4-4a88-91b9-83ac26e25e9c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:27.497189 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4747453a-d4c4-4a88-91b9-83ac26e25e9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:23:27.498788 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4747453a-d4c4-4a88-91b9-83ac26e25e9c service nova] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:23:27.518131 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 672/1360] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:23:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:23:29.740428 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5353119-5a37-4fd9-abc4-5d9c815676fe None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:29.741007 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5353119-5a37-4fd9-abc4-5d9c815676fe None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:29.741007 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5353119-5a37-4fd9-abc4-5d9c815676fe None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:29.743376 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 689/1361] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:23:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:23:29.748867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d413df24-6d5a-4227-8728-976a23125b9e None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:29.749382 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d413df24-6d5a-4227-8728-976a23125b9e None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:29.749464 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d413df24-6d5a-4227-8728-976a23125b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:29.821450 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-d413df24-6d5a-4227-8728-976a23125b9e None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:23:29.822656 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-d413df24-6d5a-4227-8728-976a23125b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:29.860998 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:29.925472 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 673/1362] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:23:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:23:29.970862 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5353119-5a37-4fd9-abc4-5d9c815676fe None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:23:29.972760 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-f5353119-5a37-4fd9-abc4-5d9c815676fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:30.146575 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5353119-5a37-4fd9-abc4-5d9c815676fe None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:23:30.157246 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:30.305563 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd392c46-1fc9-4207-93a3-c933bd9dd85e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:23:30.306356 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd392c46-1fc9-4207-93a3-c933bd9dd85e service nova] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:30.306749 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd392c46-1fc9-4207-93a3-c933bd9dd85e service nova] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:30.307133 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd392c46-1fc9-4207-93a3-c933bd9dd85e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:30.309025 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-cd392c46-1fc9-4207-93a3-c933bd9dd85e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:30.410642 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd392c46-1fc9-4207-93a3-c933bd9dd85e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:23:30.412405 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd392c46-1fc9-4207-93a3-c933bd9dd85e service nova] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:23:30.432051 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 690/1363] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:23:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:23:32.935650 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f895008-a599-4278-aaa0-c93b05ba2cee None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:32.936529 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f895008-a599-4278-aaa0-c93b05ba2cee None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:32.936529 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f895008-a599-4278-aaa0-c93b05ba2cee None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:32.937906 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 674/1364] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:23:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:23:32.943230 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd4bc1-59ef-4410-9bdf-6c3a404c9577 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:32.943675 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd4bc1-59ef-4410-9bdf-6c3a404c9577 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:32.943878 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd4bc1-59ef-4410-9bdf-6c3a404c9577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:32.960358 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:33.013722 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-b6dd4bc1-59ef-4410-9bdf-6c3a404c9577 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:23:33.014957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-b6dd4bc1-59ef-4410-9bdf-6c3a404c9577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:33.117578 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 691/1365] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:23:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:23:33.173132 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb77005-a16d-4840-b7e8-428e795e20e6 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:33.173596 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb77005-a16d-4840-b7e8-428e795e20e6 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:33.174056 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb77005-a16d-4840-b7e8-428e795e20e6 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:33.175417 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 675/1366] 10.222.0.40 () {54 vars in 962 bytes} [Fri Aug 5 15:23:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:23:33.184210 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa05de-c5a5-4c0e-8987-42357518ffa4 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:33.184210 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa05de-c5a5-4c0e-8987-42357518ffa4 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:33.184210 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa05de-c5a5-4c0e-8987-42357518ffa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:33.265065 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-42aa05de-c5a5-4c0e-8987-42357518ffa4 None None] MFA Rules not processed for user `bc0497dab1f44bb7b8196d39bf01d770`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:23:33.266525 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-42aa05de-c5a5-4c0e-8987-42357518ffa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:33.305634 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:33.400119 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 692/1367] 10.222.0.40 () {58 vars in 1059 bytes} [Fri Aug 5 15:23:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:23:33.408397 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb77005-a16d-4840-b7e8-428e795e20e6 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:23:33.410030 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-adb77005-a16d-4840-b7e8-428e795e20e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:33.564922 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb77005-a16d-4840-b7e8-428e795e20e6 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:23:33.586902 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:33.687126 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec0b510-1cd9-4fcc-b81b-48a8eeb5d242 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:23:33.688023 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0b510-1cd9-4fcc-b81b-48a8eeb5d242 service nova] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:33.688480 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0b510-1cd9-4fcc-b81b-48a8eeb5d242 service nova] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:33.688928 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0b510-1cd9-4fcc-b81b-48a8eeb5d242 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:33.690794 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-9ec0b510-1cd9-4fcc-b81b-48a8eeb5d242 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:33.794109 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec0b510-1cd9-4fcc-b81b-48a8eeb5d242 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:23:33.795720 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec0b510-1cd9-4fcc-b81b-48a8eeb5d242 service nova] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:23:33.815477 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 676/1368] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:23:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 15:23:34.737170 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a15c3-8f41-4ba4-9c8b-91cfe161d231 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:34.737708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a15c3-8f41-4ba4-9c8b-91cfe161d231 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:34.737792 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a15c3-8f41-4ba4-9c8b-91cfe161d231 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:34.739704 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 693/1369] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:23:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:23:34.745420 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa66828-2dc5-4bd6-b73c-be98c89655aa None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:34.745847 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa66828-2dc5-4bd6-b73c-be98c89655aa None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:34.746033 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa66828-2dc5-4bd6-b73c-be98c89655aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:34.781941 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:34.818062 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-5fa66828-2dc5-4bd6-b73c-be98c89655aa None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:23:34.819911 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-5fa66828-2dc5-4bd6-b73c-be98c89655aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:34.924120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 677/1370] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:23:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:23:35.262431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb6155-1954-47a3-bc54-ba063fce4804 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:35.262796 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb6155-1954-47a3-bc54-ba063fce4804 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:35.264274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb6155-1954-47a3-bc54-ba063fce4804 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:35.268114 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 694/1371] 10.222.0.40 () {54 vars in 964 bytes} [Fri Aug 5 15:23:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:23:35.270496 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be49f791-5f7c-4015-b175-735e2efed29c None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:35.270877 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be49f791-5f7c-4015-b175-735e2efed29c None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:35.271365 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be49f791-5f7c-4015-b175-735e2efed29c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:35.352643 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-be49f791-5f7c-4015-b175-735e2efed29c None None] MFA Rules not processed for user `e974f8d9a06d489884a3b00b95dd88e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:23:35.354145 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-be49f791-5f7c-4015-b175-735e2efed29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:35.402211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:35.434729 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 678/1372] 10.222.0.40 () {58 vars in 1061 bytes} [Fri Aug 5 15:23:35 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:23:35.442350 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb6155-1954-47a3-bc54-ba063fce4804 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:23:35.444017 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-22fb6155-1954-47a3-bc54-ba063fce4804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:35.561724 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb6155-1954-47a3-bc54-ba063fce4804 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:23:35.608261 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:35.642567 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a187561-f269-47f5-8d25-0571222161b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:23:35.643202 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a187561-f269-47f5-8d25-0571222161b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:35.643488 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a187561-f269-47f5-8d25-0571222161b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:35.643760 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a187561-f269-47f5-8d25-0571222161b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:35.645413 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-4a187561-f269-47f5-8d25-0571222161b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:35.771924 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a187561-f269-47f5-8d25-0571222161b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:23:35.774160 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a187561-f269-47f5-8d25-0571222161b0 service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:23:35.793691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 695/1373] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:23:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:23:36.058653 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f68cd-26af-4288-b8ca-a35c722b5fb5 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:36.059134 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f68cd-26af-4288-b8ca-a35c722b5fb5 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:36.059134 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f68cd-26af-4288-b8ca-a35c722b5fb5 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:36.060890 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 679/1374] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:23:36.065992 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27378449-11b3-4131-ab36-016254fdd212 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:36.066207 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27378449-11b3-4131-ab36-016254fdd212 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:36.066496 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27378449-11b3-4131-ab36-016254fdd212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:36.135083 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-27378449-11b3-4131-ab36-016254fdd212 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:23:36.136329 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-27378449-11b3-4131-ab36-016254fdd212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:36.158687 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:36.258897 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 696/1375] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:23:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:23:36.302308 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f68cd-26af-4288-b8ca-a35c722b5fb5 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:23:36.304638 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-347f68cd-26af-4288-b8ca-a35c722b5fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:36.451958 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f68cd-26af-4288-b8ca-a35c722b5fb5 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:23:36.503173 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:36.563261 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43528c63-688a-4be0-9bc9-aa54b79bc9bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:23:36.563919 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43528c63-688a-4be0-9bc9-aa54b79bc9bd service nova] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:36.564229 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43528c63-688a-4be0-9bc9-aa54b79bc9bd service nova] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:36.564513 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43528c63-688a-4be0-9bc9-aa54b79bc9bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:36.566326 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-43528c63-688a-4be0-9bc9-aa54b79bc9bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:36.655955 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43528c63-688a-4be0-9bc9-aa54b79bc9bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:23:36.657840 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43528c63-688a-4be0-9bc9-aa54b79bc9bd service nova] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:23:36.676761 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 680/1376] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:23:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:23:38.832905 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b975b1-7277-4f83-9c8a-e994c36f0bff None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:38.833433 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b975b1-7277-4f83-9c8a-e994c36f0bff None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:38.833568 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b975b1-7277-4f83-9c8a-e994c36f0bff None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:38.834912 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 697/1377] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:23:38.840244 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ee026-836c-4544-a116-c4ca58290427 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:38.840640 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ee026-836c-4544-a116-c4ca58290427 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:38.841063 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ee026-836c-4544-a116-c4ca58290427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:38.918407 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-770ee026-836c-4544-a116-c4ca58290427 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:23:38.919626 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-770ee026-836c-4544-a116-c4ca58290427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:38.926293 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:39.033800 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 681/1378] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:23:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:23:39.078452 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b975b1-7277-4f83-9c8a-e994c36f0bff None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:23:39.080150 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-d3b975b1-7277-4f83-9c8a-e994c36f0bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:39.234344 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b975b1-7277-4f83-9c8a-e994c36f0bff None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:23:39.238257 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:39.391557 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0f2564-ced8-495b-975e-efc661e78ec2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:23:39.392373 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f2564-ced8-495b-975e-efc661e78ec2 service nova] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:39.392782 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f2564-ced8-495b-975e-efc661e78ec2 service nova] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:39.393129 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f2564-ced8-495b-975e-efc661e78ec2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:39.394858 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-9a0f2564-ced8-495b-975e-efc661e78ec2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:39.505345 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0f2564-ced8-495b-975e-efc661e78ec2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:23:39.507250 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0f2564-ced8-495b-975e-efc661e78ec2 service nova] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:23:39.534089 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 698/1379] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:23:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:23:41.703211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e7d305-63d5-4bf6-a91f-0dad3a232ae2 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:41.703763 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e7d305-63d5-4bf6-a91f-0dad3a232ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:41.703763 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e7d305-63d5-4bf6-a91f-0dad3a232ae2 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:41.706297 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 682/1380] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:23:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:23:41.710293 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f1490-862f-4727-9ab5-39ea9ba2184a None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:41.710718 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f1490-862f-4727-9ab5-39ea9ba2184a None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:41.711335 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f1490-862f-4727-9ab5-39ea9ba2184a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:41.723494 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:41.785103 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-491f1490-862f-4727-9ab5-39ea9ba2184a None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:23:41.786475 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-491f1490-862f-4727-9ab5-39ea9ba2184a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:41.889660 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 699/1381] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:23:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:23:41.935588 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e7d305-63d5-4bf6-a91f-0dad3a232ae2 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:23:41.937366 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-26e7d305-63d5-4bf6-a91f-0dad3a232ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:42.090512 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:42.102013 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e7d305-63d5-4bf6-a91f-0dad3a232ae2 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:23:42.243904 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166fedc-e743-4bd5-951c-7d86b1dbcc34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:23:42.244687 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166fedc-e743-4bd5-951c-7d86b1dbcc34 service nova] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:42.245029 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166fedc-e743-4bd5-951c-7d86b1dbcc34 service nova] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:42.245378 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166fedc-e743-4bd5-951c-7d86b1dbcc34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:42.247168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-f166fedc-e743-4bd5-951c-7d86b1dbcc34 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:42.345853 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f166fedc-e743-4bd5-951c-7d86b1dbcc34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:23:42.347466 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f166fedc-e743-4bd5-951c-7d86b1dbcc34 service nova] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:23:42.360755 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:42.370089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 683/1382] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:23:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:23:44.635633 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34353923-26df-485a-8871-a5b066fb235d None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:44.636204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34353923-26df-485a-8871-a5b066fb235d None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:44.636369 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34353923-26df-485a-8871-a5b066fb235d None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:44.644754 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 700/1383] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:23:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:23:44.644874 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad1430c-4460-4ac0-bd59-1a924009d468 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:44.644874 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad1430c-4460-4ac0-bd59-1a924009d468 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:44.644874 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad1430c-4460-4ac0-bd59-1a924009d468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:44.710152 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-bad1430c-4460-4ac0-bd59-1a924009d468 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:23:44.711368 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-bad1430c-4460-4ac0-bd59-1a924009d468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:44.830289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 684/1384] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:23:44.874763 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34353923-26df-485a-8871-a5b066fb235d None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:23:44.876406 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-34353923-26df-485a-8871-a5b066fb235d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:44.965759 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:45.028749 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34353923-26df-485a-8871-a5b066fb235d None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:23:45.143637 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cbf547-969b-47a6-be0a-af5113984f48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:23:45.144294 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbf547-969b-47a6-be0a-af5113984f48 service nova] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:45.144583 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbf547-969b-47a6-be0a-af5113984f48 service nova] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:45.144861 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbf547-969b-47a6-be0a-af5113984f48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:45.146507 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-86cbf547-969b-47a6-be0a-af5113984f48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:45.207148 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:45.248880 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cbf547-969b-47a6-be0a-af5113984f48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:23:45.250381 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cbf547-969b-47a6-be0a-af5113984f48 service nova] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:23:45.269614 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 701/1385] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:23:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:23:47.260360 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021427c-d852-4500-b731-0d67f89bd4fb None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:47.260897 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021427c-d852-4500-b731-0d67f89bd4fb None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:47.260897 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021427c-d852-4500-b731-0d67f89bd4fb None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:47.262288 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 685/1386] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:23:47.269046 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a58ee5-5d62-433b-bb08-8ddc1effcaec None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:47.269679 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a58ee5-5d62-433b-bb08-8ddc1effcaec None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:47.269679 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a58ee5-5d62-433b-bb08-8ddc1effcaec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:47.340638 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-62a58ee5-5d62-433b-bb08-8ddc1effcaec None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:23:47.341921 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-62a58ee5-5d62-433b-bb08-8ddc1effcaec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:47.445026 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 702/1387] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:23:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:23:47.491370 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021427c-d852-4500-b731-0d67f89bd4fb None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:23:47.493546 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-6021427c-d852-4500-b731-0d67f89bd4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:47.607461 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:47.687777 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021427c-d852-4500-b731-0d67f89bd4fb None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:23:47.804257 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e61cffb-517a-4506-b12d-9ed5faea7011 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:23:47.804888 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61cffb-517a-4506-b12d-9ed5faea7011 service nova] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:47.805172 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61cffb-517a-4506-b12d-9ed5faea7011 service nova] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:47.805447 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61cffb-517a-4506-b12d-9ed5faea7011 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:47.807117 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-3e61cffb-517a-4506-b12d-9ed5faea7011 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:47.845824 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:47.905998 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e61cffb-517a-4506-b12d-9ed5faea7011 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:23:47.908219 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e61cffb-517a-4506-b12d-9ed5faea7011 service nova] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:23:47.932657 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 686/1388] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:23:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:23:50.125157 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a697309-499d-47b2-b7f4-209ca9268fee None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:50.125699 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a697309-499d-47b2-b7f4-209ca9268fee None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:50.125699 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a697309-499d-47b2-b7f4-209ca9268fee None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:50.127337 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 703/1389] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:23:50.132493 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a2cf4-01a6-4046-bb1b-fae94c2e835a None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:50.132910 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a2cf4-01a6-4046-bb1b-fae94c2e835a None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:50.133057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a2cf4-01a6-4046-bb1b-fae94c2e835a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:50.205295 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-2b9a2cf4-01a6-4046-bb1b-fae94c2e835a None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:23:50.206611 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-2b9a2cf4-01a6-4046-bb1b-fae94c2e835a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:50.305147 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 687/1390] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:23:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:23:50.349088 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a697309-499d-47b2-b7f4-209ca9268fee None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:23:50.351301 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-5a697309-499d-47b2-b7f4-209ca9268fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:50.382159 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:50.499180 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a697309-499d-47b2-b7f4-209ca9268fee None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:23:50.602272 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:50.613999 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1789859-bdfa-489e-b49b-18e020b00383 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:23:50.614623 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1789859-bdfa-489e-b49b-18e020b00383 service nova] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:50.614910 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1789859-bdfa-489e-b49b-18e020b00383 service nova] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:50.615190 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1789859-bdfa-489e-b49b-18e020b00383 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:50.616884 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-d1789859-bdfa-489e-b49b-18e020b00383 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:50.708440 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1789859-bdfa-489e-b49b-18e020b00383 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:23:50.710486 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1789859-bdfa-489e-b49b-18e020b00383 service nova] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:23:50.729264 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 704/1391] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:23:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:23:51.046397 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82a422-3939-4394-baba-af66373d5c2e None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:51.046823 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82a422-3939-4394-baba-af66373d5c2e None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:51.047201 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82a422-3939-4394-baba-af66373d5c2e None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:51.048750 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 688/1392] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:23:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:23:51.054443 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be519f-e71c-48cf-8b4c-05ed9d0654f3 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:51.054826 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be519f-e71c-48cf-8b4c-05ed9d0654f3 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:51.055176 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be519f-e71c-48cf-8b4c-05ed9d0654f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:51.122185 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-60be519f-e71c-48cf-8b4c-05ed9d0654f3 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:23:51.123416 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-60be519f-e71c-48cf-8b4c-05ed9d0654f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:51.183280 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:51.222193 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 705/1393] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:23:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:23:51.279380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ed3e34-5fee-44f7-927a-8dad09320091 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:51.279798 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ed3e34-5fee-44f7-927a-8dad09320091 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:51.279918 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ed3e34-5fee-44f7-927a-8dad09320091 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:51.281324 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 689/1394] 10.222.0.40 () {54 vars in 966 bytes} [Fri Aug 5 15:23:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:23:51.286263 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2881f-8f61-42be-9191-09a7b6291c0a None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:51.286560 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2881f-8f61-42be-9191-09a7b6291c0a None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:51.286837 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2881f-8f61-42be-9191-09a7b6291c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:51.364246 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-93e2881f-8f61-42be-9191-09a7b6291c0a None None] MFA Rules not processed for user `c386fa0483cf42efaddbc2395f6d1fce`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:23:51.365923 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-93e2881f-8f61-42be-9191-09a7b6291c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:51.440774 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 706/1395] 10.222.0.40 () {58 vars in 1063 bytes} [Fri Aug 5 15:23:51 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:23:51.448894 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ed3e34-5fee-44f7-927a-8dad09320091 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:23:51.450406 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-c3ed3e34-5fee-44f7-927a-8dad09320091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:51.463956 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:51.554331 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ed3e34-5fee-44f7-927a-8dad09320091 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:23:51.630885 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b263331-1f26-4373-bfc0-3e75d0f13af4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c386fa0483cf42efaddbc2395f6d1fce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:23:51.631424 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b263331-1f26-4373-bfc0-3e75d0f13af4 service glance] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:51.631709 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b263331-1f26-4373-bfc0-3e75d0f13af4 service glance] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:51.632013 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b263331-1f26-4373-bfc0-3e75d0f13af4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:51.633691 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-1b263331-1f26-4373-bfc0-3e75d0f13af4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:51.703474 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:51.745137 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b263331-1f26-4373-bfc0-3e75d0f13af4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:23:51.746577 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b263331-1f26-4373-bfc0-3e75d0f13af4 service glance] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:23:51.764476 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 690/1396] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:23:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:23:52.710979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e3f80-144b-4d84-98be-ec1e6e2bf281 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:52.711472 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e3f80-144b-4d84-98be-ec1e6e2bf281 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:52.711570 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e3f80-144b-4d84-98be-ec1e6e2bf281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:52.777622 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-655e3f80-144b-4d84-98be-ec1e6e2bf281 None None] MFA Rules not processed for user `1654fb6a81e14d6b99ab495bb6d029d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:23:52.778825 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-655e3f80-144b-4d84-98be-ec1e6e2bf281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:52.787230 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:52.868440 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 707/1397] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 5 15:23:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:23:52.912825 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8b14c-681b-4914-8955-b72a052a2bdd None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:52.913312 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004bd2c-c693-4e86-936f-8b3cf337adf6 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:52.913312 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004bd2c-c693-4e86-936f-8b3cf337adf6 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:52.915199 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8b14c-681b-4914-8955-b72a052a2bdd None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:52.915294 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004bd2c-c693-4e86-936f-8b3cf337adf6 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:52.915382 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8b14c-681b-4914-8955-b72a052a2bdd None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:52.915479 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 708/1399] 10.222.0.40 () {54 vars in 943 bytes} [Fri Aug 5 15:23:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:23:52.915598 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 691/1399] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:23:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:23:52.920666 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ed9d5-3d2e-47de-81f2-0141739141d4 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:52.921160 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ed9d5-3d2e-47de-81f2-0141739141d4 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:52.921274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ed9d5-3d2e-47de-81f2-0141739141d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:52.928152 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407719f8-1aa5-469c-98be-5e9c0c2c472e None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:52.928446 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407719f8-1aa5-469c-98be-5e9c0c2c472e None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:52.928733 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407719f8-1aa5-469c-98be-5e9c0c2c472e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:52.992129 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-135ed9d5-3d2e-47de-81f2-0141739141d4 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:23:52.993298 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-135ed9d5-3d2e-47de-81f2-0141739141d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:53.008602 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-407719f8-1aa5-469c-98be-5e9c0c2c472e None None] MFA Rules not processed for user `85154ce7446245738edcc8702dbf6861`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:23:53.009883 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-407719f8-1aa5-469c-98be-5e9c0c2c472e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:53.093216 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 709/1400] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:23:53.119143 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 692/1401] 10.222.0.40 () {58 vars in 1040 bytes} [Fri Aug 5 15:23:52 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:23:53.129139 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ed9d5-3d2e-47de-81f2-0141739141d4 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:23:53.130355 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-135ed9d5-3d2e-47de-81f2-0141739141d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:53.133068 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:53.138644 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407719f8-1aa5-469c-98be-5e9c0c2c472e None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:23:53.139854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-407719f8-1aa5-469c-98be-5e9c0c2c472e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:53.142975 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:53.237862 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407719f8-1aa5-469c-98be-5e9c0c2c472e None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:23:53.239196 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ed9d5-3d2e-47de-81f2-0141739141d4 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:23:53.344890 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a050bc-116a-4cb9-83f1-dad9b15cd1de service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85154ce7446245738edcc8702dbf6861', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:23:53.345511 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a050bc-116a-4cb9-83f1-dad9b15cd1de service swift] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:53.345792 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a050bc-116a-4cb9-83f1-dad9b15cd1de service swift] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:53.346061 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a050bc-116a-4cb9-83f1-dad9b15cd1de service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:53.347732 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-15a050bc-116a-4cb9-83f1-dad9b15cd1de service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:53.351936 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd4f910-6e86-423b-b8ed-c20dc7b839db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:23:53.352588 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4f910-6e86-423b-b8ed-c20dc7b839db service nova] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:53.352873 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4f910-6e86-423b-b8ed-c20dc7b839db service nova] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:53.353146 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4f910-6e86-423b-b8ed-c20dc7b839db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:53.354758 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-cdd4f910-6e86-423b-b8ed-c20dc7b839db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:53.367355 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:53.370267 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:53.440709 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a050bc-116a-4cb9-83f1-dad9b15cd1de service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:23:53.442522 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a050bc-116a-4cb9-83f1-dad9b15cd1de service swift] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:23:53.451605 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 710/1402] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 5 15:23:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 15:23:53.454166 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd4f910-6e86-423b-b8ed-c20dc7b839db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:23:53.455868 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd4f910-6e86-423b-b8ed-c20dc7b839db service nova] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:23:53.477189 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 693/1403] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:23:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 15:23:55.702184 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9f464-778a-4112-aa50-de360cbb9513 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:55.702832 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9f464-778a-4112-aa50-de360cbb9513 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:55.702832 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9f464-778a-4112-aa50-de360cbb9513 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:55.704274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 711/1404] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:23:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:23:55.710469 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f0958-e20e-409d-b980-8b4125d43930 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:55.710922 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f0958-e20e-409d-b980-8b4125d43930 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:55.711011 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f0958-e20e-409d-b980-8b4125d43930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:55.789750 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-e58f0958-e20e-409d-b980-8b4125d43930 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:23:55.791231 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-e58f0958-e20e-409d-b980-8b4125d43930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:55.901550 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:55.914448 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 694/1405] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:23:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:23:55.959944 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d9f464-778a-4112-aa50-de360cbb9513 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:23:55.961064 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-82d9f464-778a-4112-aa50-de360cbb9513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:56.125539 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d9f464-778a-4112-aa50-de360cbb9513 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:23:56.143500 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:56.263841 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8218ab1-45e7-4b48-8005-63dde0dcd1f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:23:56.264494 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8218ab1-45e7-4b48-8005-63dde0dcd1f5 service nova] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:56.264775 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8218ab1-45e7-4b48-8005-63dde0dcd1f5 service nova] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:56.265047 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8218ab1-45e7-4b48-8005-63dde0dcd1f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:56.266697 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-d8218ab1-45e7-4b48-8005-63dde0dcd1f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:56.371578 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8218ab1-45e7-4b48-8005-63dde0dcd1f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:23:56.373261 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8218ab1-45e7-4b48-8005-63dde0dcd1f5 service nova] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:23:56.393557 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 712/1406] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:23:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:23:58.611843 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c360c4c-e064-432b-9133-fa2b7efeceb2 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:58.613024 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c360c4c-e064-432b-9133-fa2b7efeceb2 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:58.613406 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c360c4c-e064-432b-9133-fa2b7efeceb2 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:58.614919 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 695/1407] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:23:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:23:58.620538 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea8a29-a0f3-4a26-a728-fcaf185c08f1 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:58.620991 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea8a29-a0f3-4a26-a728-fcaf185c08f1 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:58.621136 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea8a29-a0f3-4a26-a728-fcaf185c08f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:58.651336 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:58.696899 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-76ea8a29-a0f3-4a26-a728-fcaf185c08f1 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:23:58.698268 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-76ea8a29-a0f3-4a26-a728-fcaf185c08f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:58.816001 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 713/1408] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:23:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:23:58.864183 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c360c4c-e064-432b-9133-fa2b7efeceb2 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:23:58.864362 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-6c360c4c-e064-432b-9133-fa2b7efeceb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:59.030654 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c360c4c-e064-432b-9133-fa2b7efeceb2 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:23:59.103125 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:23:59.161336 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103440ed-5996-4f70-9887-4f4168ad894a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:23:59.161888 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103440ed-5996-4f70-9887-4f4168ad894a service nova] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:23:59.162167 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103440ed-5996-4f70-9887-4f4168ad894a service nova] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:23:59.162443 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103440ed-5996-4f70-9887-4f4168ad894a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:23:59.164827 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-103440ed-5996-4f70-9887-4f4168ad894a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:23:59.263951 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103440ed-5996-4f70-9887-4f4168ad894a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:23:59.265600 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103440ed-5996-4f70-9887-4f4168ad894a service nova] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:23:59.287011 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 696/1409] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:23:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:24:08.814302 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=56 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:09.284236 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=66 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:17.017837 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e468b82-19ab-41ba-8422-adc359255e29 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:17.018409 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e468b82-19ab-41ba-8422-adc359255e29 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:17.018409 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e468b82-19ab-41ba-8422-adc359255e29 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:17.019916 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 714/1410] 10.222.0.40 () {54 vars in 918 bytes} [Fri Aug 5 15:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:24:17.025723 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df9835-7c36-4efd-8d23-3f8d74b22801 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:17.026133 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df9835-7c36-4efd-8d23-3f8d74b22801 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:17.026265 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df9835-7c36-4efd-8d23-3f8d74b22801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:17.099778 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-20df9835-7c36-4efd-8d23-3f8d74b22801 None None] MFA Rules not processed for user `107a7c18a2c6440b86e1b7c3e39cc724`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:24:17.101012 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-20df9835-7c36-4efd-8d23-3f8d74b22801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:17.207369 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 697/1411] 10.222.0.40 () {58 vars in 1015 bytes} [Fri Aug 5 15:24:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:24:17.222344 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c94ffc-67b8-4417-95a5-63e70057fc9e None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:17.222636 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c94ffc-67b8-4417-95a5-63e70057fc9e None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:17.222916 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c94ffc-67b8-4417-95a5-63e70057fc9e None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:17.224334 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 715/1412] 10.222.0.40 () {54 vars in 918 bytes} [Fri Aug 5 15:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:24:17.228896 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a35ccb-6239-46fc-a83c-b599ac105aa6 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:17.229190 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a35ccb-6239-46fc-a83c-b599ac105aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:17.229471 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a35ccb-6239-46fc-a83c-b599ac105aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:17.302767 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:17.306248 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-b9a35ccb-6239-46fc-a83c-b599ac105aa6 None None] MFA Rules not processed for user `bc0497dab1f44bb7b8196d39bf01d770`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:24:17.307479 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-b9a35ccb-6239-46fc-a83c-b599ac105aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:17.423324 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 698/1413] 10.222.0.40 () {58 vars in 1015 bytes} [Fri Aug 5 15:24:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:24:19.953700 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d5189-7bd9-451f-9c53-5136dc8abc1a None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:19.954208 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d5189-7bd9-451f-9c53-5136dc8abc1a None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:19.954208 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d5189-7bd9-451f-9c53-5136dc8abc1a None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:19.955530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 716/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 15:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:24:19.961282 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e73179-17f3-4d40-9b12-c47bd37f1de9 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:19.961673 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e73179-17f3-4d40-9b12-c47bd37f1de9 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:19.961840 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e73179-17f3-4d40-9b12-c47bd37f1de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:20.027964 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-03e73179-17f3-4d40-9b12-c47bd37f1de9 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:24:20.029212 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-03e73179-17f3-4d40-9b12-c47bd37f1de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:20.082619 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:20.124376 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 699/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 15:24:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:24:20.167929 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d5189-7bd9-451f-9c53-5136dc8abc1a None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:24:20.169567 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-5d0d5189-7bd9-451f-9c53-5136dc8abc1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:20.320511 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d5189-7bd9-451f-9c53-5136dc8abc1a None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:24:20.393391 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:20.434258 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dc2d82-0bab-4ce9-9096-e7bbe90030ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:24:20.434875 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc2d82-0bab-4ce9-9096-e7bbe90030ca service nova] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:20.435162 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc2d82-0bab-4ce9-9096-e7bbe90030ca service nova] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:20.435440 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc2d82-0bab-4ce9-9096-e7bbe90030ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:20.437100 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-89dc2d82-0bab-4ce9-9096-e7bbe90030ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:20.525849 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dc2d82-0bab-4ce9-9096-e7bbe90030ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:24:20.527461 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dc2d82-0bab-4ce9-9096-e7bbe90030ca service nova] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:24:20.545299 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 717/1416] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:24:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:24:21.435234 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734217c-0e8e-4094-9a22-da4f85bd8b33 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:21.435760 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734217c-0e8e-4094-9a22-da4f85bd8b33 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:21.435760 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734217c-0e8e-4094-9a22-da4f85bd8b33 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:21.437091 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 700/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 15:24:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:24:21.441933 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a94f2-8ec4-47ee-8912-4398a9d54a44 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:21.442378 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a94f2-8ec4-47ee-8912-4398a9d54a44 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:21.442485 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a94f2-8ec4-47ee-8912-4398a9d54a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:21.519048 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-f84a94f2-8ec4-47ee-8912-4398a9d54a44 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:24:21.520275 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-f84a94f2-8ec4-47ee-8912-4398a9d54a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:21.547145 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:21.615627 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 718/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 15:24:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:24:21.767192 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6734217c-0e8e-4094-9a22-da4f85bd8b33 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:24:21.768743 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-6734217c-0e8e-4094-9a22-da4f85bd8b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:21.914217 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6734217c-0e8e-4094-9a22-da4f85bd8b33 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:24:21.954096 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:22.026345 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61dd9ef-10b4-4487-90c1-52a877f7af91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:24:22.026923 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61dd9ef-10b4-4487-90c1-52a877f7af91 service nova] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:22.027212 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61dd9ef-10b4-4487-90c1-52a877f7af91 service nova] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:22.027495 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61dd9ef-10b4-4487-90c1-52a877f7af91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:22.029175 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-a61dd9ef-10b4-4487-90c1-52a877f7af91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:22.117263 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61dd9ef-10b4-4487-90c1-52a877f7af91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:24:22.118869 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61dd9ef-10b4-4487-90c1-52a877f7af91 service nova] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:24:22.137455 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 701/1419] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:24:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:24:23.018507 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1fafcd-3de6-46bd-b470-d75242aeddd8 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:23.019046 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1fafcd-3de6-46bd-b470-d75242aeddd8 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:23.019046 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1fafcd-3de6-46bd-b470-d75242aeddd8 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:23.020496 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 719/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 15:24:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:24:23.026264 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2e111-fe60-4a3d-a3fe-ae0852fa12e0 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:23.026705 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2e111-fe60-4a3d-a3fe-ae0852fa12e0 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:23.026836 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2e111-fe60-4a3d-a3fe-ae0852fa12e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:23.081479 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:23.224559 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-6cd2e111-fe60-4a3d-a3fe-ae0852fa12e0 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:24:23.225842 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-6cd2e111-fe60-4a3d-a3fe-ae0852fa12e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:23.323164 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 702/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 15:24:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:24:23.476152 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1fafcd-3de6-46bd-b470-d75242aeddd8 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:24:23.477646 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-2a1fafcd-3de6-46bd-b470-d75242aeddd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:23.621092 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1fafcd-3de6-46bd-b470-d75242aeddd8 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:24:23.633673 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:23.733590 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756c859c-08af-4c56-a9b4-756b2c82051e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:24:23.734216 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c859c-08af-4c56-a9b4-756b2c82051e service nova] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:23.734505 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c859c-08af-4c56-a9b4-756b2c82051e service nova] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:23.734786 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c859c-08af-4c56-a9b4-756b2c82051e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:23.736451 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-756c859c-08af-4c56-a9b4-756b2c82051e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:23.824932 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756c859c-08af-4c56-a9b4-756b2c82051e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:24:23.826516 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756c859c-08af-4c56-a9b4-756b2c82051e service nova] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:24:23.844161 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 720/1422] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:24:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:24:24.710875 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92666bb5-5826-44b0-8ac0-5f8aca817329 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:24.711445 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92666bb5-5826-44b0-8ac0-5f8aca817329 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:24.711445 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92666bb5-5826-44b0-8ac0-5f8aca817329 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:24.712828 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 703/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 15:24:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:24:24.717702 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b436fedc-3847-4472-8fe3-c4c7c36622a6 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:24.718093 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b436fedc-3847-4472-8fe3-c4c7c36622a6 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:24.718289 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b436fedc-3847-4472-8fe3-c4c7c36622a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:24.738572 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:24.796547 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-b436fedc-3847-4472-8fe3-c4c7c36622a6 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:24:24.797761 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-b436fedc-3847-4472-8fe3-c4c7c36622a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:24.893349 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 721/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 15:24:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:24:25.041866 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92666bb5-5826-44b0-8ac0-5f8aca817329 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:24:25.043406 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-92666bb5-5826-44b0-8ac0-5f8aca817329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:25.151389 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:25.189430 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92666bb5-5826-44b0-8ac0-5f8aca817329 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:24:25.299945 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b6b5b4-0318-4fc4-8f51-9cf3d2bf9835 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:24:25.300514 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b6b5b4-0318-4fc4-8f51-9cf3d2bf9835 service nova] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:25.300797 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b6b5b4-0318-4fc4-8f51-9cf3d2bf9835 service nova] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:25.301068 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b6b5b4-0318-4fc4-8f51-9cf3d2bf9835 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:25.302689 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-41b6b5b4-0318-4fc4-8f51-9cf3d2bf9835 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:25.376921 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:25.389857 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b6b5b4-0318-4fc4-8f51-9cf3d2bf9835 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:24:25.391470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b6b5b4-0318-4fc4-8f51-9cf3d2bf9835 service nova] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:24:25.409706 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 704/1425] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:24:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:24:26.298194 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20438455-9e33-4e33-b644-4715e4f2c3f7 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:26.298760 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20438455-9e33-4e33-b644-4715e4f2c3f7 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:26.298760 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20438455-9e33-4e33-b644-4715e4f2c3f7 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:26.300153 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 722/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 15:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:24:26.305796 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883cabad-a61f-40a6-9df3-daefa1a81fa4 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:26.306242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883cabad-a61f-40a6-9df3-daefa1a81fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:26.306342 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883cabad-a61f-40a6-9df3-daefa1a81fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:26.380928 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-883cabad-a61f-40a6-9df3-daefa1a81fa4 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:24:26.382777 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-883cabad-a61f-40a6-9df3-daefa1a81fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:26.482115 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 705/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 15:24:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:24:26.636418 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20438455-9e33-4e33-b644-4715e4f2c3f7 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:24:26.637959 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-20438455-9e33-4e33-b644-4715e4f2c3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:26.729082 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:26.796563 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20438455-9e33-4e33-b644-4715e4f2c3f7 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:24:26.921380 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9087691-8d1d-4d06-8651-393303a79a76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:24:26.921998 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9087691-8d1d-4d06-8651-393303a79a76 service nova] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:26.922280 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9087691-8d1d-4d06-8651-393303a79a76 service nova] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:26.922552 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9087691-8d1d-4d06-8651-393303a79a76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:26.924215 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-f9087691-8d1d-4d06-8651-393303a79a76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:26.975892 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:27.017555 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9087691-8d1d-4d06-8651-393303a79a76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:24:27.019168 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9087691-8d1d-4d06-8651-393303a79a76 service nova] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:24:27.039125 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 723/1428] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:24:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:24:27.904030 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a355c1a-8c52-45f0-9a52-6415d0ea46bb None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:27.904716 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a355c1a-8c52-45f0-9a52-6415d0ea46bb None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:27.904716 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a355c1a-8c52-45f0-9a52-6415d0ea46bb None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:27.905975 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 706/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 15:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:24:27.910860 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9daba-2b91-45fd-bb6d-bb2b2779a272 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:27.911288 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9daba-2b91-45fd-bb6d-bb2b2779a272 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:27.911421 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9daba-2b91-45fd-bb6d-bb2b2779a272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:28.106559 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-10b9daba-2b91-45fd-bb6d-bb2b2779a272 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:24:28.107806 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-10b9daba-2b91-45fd-bb6d-bb2b2779a272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:28.210312 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 724/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 15:24:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:24:28.362710 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a355c1a-8c52-45f0-9a52-6415d0ea46bb None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:24:28.364284 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-5a355c1a-8c52-45f0-9a52-6415d0ea46bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:28.413303 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:28.520955 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a355c1a-8c52-45f0-9a52-6415d0ea46bb None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:24:28.630963 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d97c7a-2b3f-4b15-baf2-07d567a452b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:24:28.632181 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d97c7a-2b3f-4b15-baf2-07d567a452b2 service nova] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:28.632459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d97c7a-2b3f-4b15-baf2-07d567a452b2 service nova] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:28.632728 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d97c7a-2b3f-4b15-baf2-07d567a452b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:28.634349 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-51d97c7a-2b3f-4b15-baf2-07d567a452b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:28.641129 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:28.723848 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d97c7a-2b3f-4b15-baf2-07d567a452b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:24:28.725513 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d97c7a-2b3f-4b15-baf2-07d567a452b2 service nova] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:24:28.743960 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 707/1431] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:24:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:24:29.650787 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a45aa3-5f9a-4a63-b01f-8d14222c3666 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:29.651336 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a45aa3-5f9a-4a63-b01f-8d14222c3666 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:29.651336 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a45aa3-5f9a-4a63-b01f-8d14222c3666 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:29.657626 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 725/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 15:24:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:24:29.657940 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b7435-61f3-4ece-8bcb-aaf496f16361 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:29.658364 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b7435-61f3-4ece-8bcb-aaf496f16361 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:29.658504 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b7435-61f3-4ece-8bcb-aaf496f16361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:29.733077 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-fe8b7435-61f3-4ece-8bcb-aaf496f16361 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:24:29.734332 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-fe8b7435-61f3-4ece-8bcb-aaf496f16361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:29.802171 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:29.832055 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 708/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 15:24:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:24:29.982460 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a45aa3-5f9a-4a63-b01f-8d14222c3666 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:24:29.983972 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-f6a45aa3-5f9a-4a63-b01f-8d14222c3666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:30.001951 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:30.127010 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a45aa3-5f9a-4a63-b01f-8d14222c3666 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:24:30.218935 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:30.252009 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a149ace-6015-4985-9ba7-2d0d470fcc24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:24:30.252665 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a149ace-6015-4985-9ba7-2d0d470fcc24 service nova] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:30.252948 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a149ace-6015-4985-9ba7-2d0d470fcc24 service nova] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:30.253223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a149ace-6015-4985-9ba7-2d0d470fcc24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:30.255564 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-0a149ace-6015-4985-9ba7-2d0d470fcc24 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:30.343296 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a149ace-6015-4985-9ba7-2d0d470fcc24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:24:30.344928 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a149ace-6015-4985-9ba7-2d0d470fcc24 service nova] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:24:30.362490 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 726/1434] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:24:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:24:31.206102 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d518b-37b1-450f-bffa-3a3b9aa52812 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:31.206589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d518b-37b1-450f-bffa-3a3b9aa52812 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:31.206687 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d518b-37b1-450f-bffa-3a3b9aa52812 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:31.208022 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 709/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 15:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:24:31.213525 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b44cf-1f06-4357-9474-88079a24f243 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:31.213977 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b44cf-1f06-4357-9474-88079a24f243 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:31.214081 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b44cf-1f06-4357-9474-88079a24f243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:31.290510 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-961b44cf-1f06-4357-9474-88079a24f243 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:24:31.291723 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-961b44cf-1f06-4357-9474-88079a24f243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:31.332074 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:31.386409 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 727/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 15:24:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:24:31.537811 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9d518b-37b1-450f-bffa-3a3b9aa52812 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:24:31.539403 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-cb9d518b-37b1-450f-bffa-3a3b9aa52812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:31.685271 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9d518b-37b1-450f-bffa-3a3b9aa52812 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:24:31.738452 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:31.797094 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2ac6db-f03a-48da-9c5c-af29de82cd9e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:24:31.797632 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ac6db-f03a-48da-9c5c-af29de82cd9e service nova] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:31.797911 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ac6db-f03a-48da-9c5c-af29de82cd9e service nova] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:31.798183 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ac6db-f03a-48da-9c5c-af29de82cd9e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:31.800153 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-8e2ac6db-f03a-48da-9c5c-af29de82cd9e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:31.889137 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2ac6db-f03a-48da-9c5c-af29de82cd9e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:24:31.890754 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2ac6db-f03a-48da-9c5c-af29de82cd9e service nova] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:24:31.908147 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 710/1437] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:24:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:24:32.757040 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10faa5ec-74e0-4f69-8c2c-e87bcd557a5b None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:32.757625 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10faa5ec-74e0-4f69-8c2c-e87bcd557a5b None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:32.757625 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10faa5ec-74e0-4f69-8c2c-e87bcd557a5b None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:32.758961 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 728/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 15:24:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:24:32.763814 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9a238-ec9e-467a-ab15-7976aa7b6df0 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:32.764269 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9a238-ec9e-467a-ab15-7976aa7b6df0 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:32.764408 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9a238-ec9e-467a-ab15-7976aa7b6df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:32.839128 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-f8a9a238-ec9e-467a-ab15-7976aa7b6df0 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:24:32.840303 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-f8a9a238-ec9e-467a-ab15-7976aa7b6df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:32.842300 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:32.937015 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 711/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 15:24:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:24:33.089597 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10faa5ec-74e0-4f69-8c2c-e87bcd557a5b None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:24:33.091090 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-10faa5ec-74e0-4f69-8c2c-e87bcd557a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:33.236896 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10faa5ec-74e0-4f69-8c2c-e87bcd557a5b None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:24:33.263835 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:33.348009 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f01e3d1-fe57-42aa-8aca-e4cbd729d6d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:24:33.348591 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01e3d1-fe57-42aa-8aca-e4cbd729d6d7 service nova] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:33.348872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01e3d1-fe57-42aa-8aca-e4cbd729d6d7 service nova] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:33.349147 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01e3d1-fe57-42aa-8aca-e4cbd729d6d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:33.351368 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-0f01e3d1-fe57-42aa-8aca-e4cbd729d6d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:33.438083 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f01e3d1-fe57-42aa-8aca-e4cbd729d6d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:24:33.439671 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f01e3d1-fe57-42aa-8aca-e4cbd729d6d7 service nova] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:24:33.457900 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 729/1440] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:24:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:24:35.410706 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199cb7e-955c-4ff8-a741-9ed7d26c3bca None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:35.411269 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199cb7e-955c-4ff8-a741-9ed7d26c3bca None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:35.411269 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199cb7e-955c-4ff8-a741-9ed7d26c3bca None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:35.412901 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 712/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 15:24:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:24:35.418305 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85833c33-a4a8-40b2-bd1d-d2acd10da90a None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:35.418701 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85833c33-a4a8-40b2-bd1d-d2acd10da90a None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:35.418864 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85833c33-a4a8-40b2-bd1d-d2acd10da90a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:35.461287 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:35.485502 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-85833c33-a4a8-40b2-bd1d-d2acd10da90a None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:24:35.486696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-85833c33-a4a8-40b2-bd1d-d2acd10da90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:35.583361 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 730/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 15:24:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:24:35.621959 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2199cb7e-955c-4ff8-a741-9ed7d26c3bca None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:24:35.623508 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-2199cb7e-955c-4ff8-a741-9ed7d26c3bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:35.726811 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2199cb7e-955c-4ff8-a741-9ed7d26c3bca None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:24:35.758769 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:35.799032 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692cc7a6-5e77-4b7c-8677-e64d11f2042e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:24:35.799613 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692cc7a6-5e77-4b7c-8677-e64d11f2042e service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:35.799893 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692cc7a6-5e77-4b7c-8677-e64d11f2042e service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:35.800199 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692cc7a6-5e77-4b7c-8677-e64d11f2042e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:35.801806 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-692cc7a6-5e77-4b7c-8677-e64d11f2042e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:35.909433 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692cc7a6-5e77-4b7c-8677-e64d11f2042e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:24:35.910867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692cc7a6-5e77-4b7c-8677-e64d11f2042e service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:24:35.931465 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 713/1443] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:24:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:24:38.075967 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60edfc-7f96-4db6-b0ec-16354ab204bf None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:38.076599 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60edfc-7f96-4db6-b0ec-16354ab204bf None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:38.076599 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60edfc-7f96-4db6-b0ec-16354ab204bf None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:38.077905 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 731/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 15:24:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:24:38.083056 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a5c6a-e8fd-4a61-b742-122c857e3b61 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:38.083463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a5c6a-e8fd-4a61-b742-122c857e3b61 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:38.083613 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a5c6a-e8fd-4a61-b742-122c857e3b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:38.148219 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-b44a5c6a-e8fd-4a61-b742-122c857e3b61 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:24:38.149426 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-b44a5c6a-e8fd-4a61-b742-122c857e3b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:38.161858 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:38.256596 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 714/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 15:24:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:24:38.294904 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d60edfc-7f96-4db6-b0ec-16354ab204bf None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:24:38.296475 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-8d60edfc-7f96-4db6-b0ec-16354ab204bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:38.398243 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d60edfc-7f96-4db6-b0ec-16354ab204bf None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:24:38.410000 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:38.470079 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb39a3-741d-4d64-9996-44b9b69438c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:24:38.470693 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb39a3-741d-4d64-9996-44b9b69438c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:38.470976 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb39a3-741d-4d64-9996-44b9b69438c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:38.471253 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb39a3-741d-4d64-9996-44b9b69438c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:38.472896 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-bcdb39a3-741d-4d64-9996-44b9b69438c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:38.580903 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdb39a3-741d-4d64-9996-44b9b69438c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:24:38.582340 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdb39a3-741d-4d64-9996-44b9b69438c3 service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:24:38.601270 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 732/1446] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:24:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:24:41.048719 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c3299-4c05-4787-96b6-7a25671f197a None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:41.049316 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c3299-4c05-4787-96b6-7a25671f197a None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:41.049316 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c3299-4c05-4787-96b6-7a25671f197a None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:41.050640 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 715/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 15:24:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:24:41.056095 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf25a85-2b8c-4672-af3d-99fe7214adc5 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:41.056497 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf25a85-2b8c-4672-af3d-99fe7214adc5 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:41.056660 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf25a85-2b8c-4672-af3d-99fe7214adc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:41.104211 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:41.121032 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-bbf25a85-2b8c-4672-af3d-99fe7214adc5 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:24:41.122273 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-bbf25a85-2b8c-4672-af3d-99fe7214adc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:41.226381 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 733/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 15:24:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:24:41.266650 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c3299-4c05-4787-96b6-7a25671f197a None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:24:41.268279 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-a11c3299-4c05-4787-96b6-7a25671f197a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:41.370985 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c3299-4c05-4787-96b6-7a25671f197a None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:24:41.410165 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:41.444470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00480441-851c-4f28-adb6-9631be6543a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:24:41.445083 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00480441-851c-4f28-adb6-9631be6543a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:41.445383 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00480441-851c-4f28-adb6-9631be6543a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:41.445644 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00480441-851c-4f28-adb6-9631be6543a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:41.447275 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-00480441-851c-4f28-adb6-9631be6543a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:41.553745 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00480441-851c-4f28-adb6-9631be6543a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:24:41.555180 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00480441-851c-4f28-adb6-9631be6543a6 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:24:41.573980 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 716/1449] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:24:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:24:43.684079 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b50605-b880-448e-87fd-85f52526aeba None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:43.684646 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b50605-b880-448e-87fd-85f52526aeba None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:43.684646 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b50605-b880-448e-87fd-85f52526aeba None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:43.686007 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 734/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 15:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:24:43.691399 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbcfca3-0a0a-4b33-9cda-356ba15078ef None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:43.691808 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbcfca3-0a0a-4b33-9cda-356ba15078ef None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:43.691949 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbcfca3-0a0a-4b33-9cda-356ba15078ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:43.758115 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-8bbcfca3-0a0a-4b33-9cda-356ba15078ef None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:24:43.759318 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-8bbcfca3-0a0a-4b33-9cda-356ba15078ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:43.780481 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:43.856196 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 717/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 15:24:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:24:43.896498 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b50605-b880-448e-87fd-85f52526aeba None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:24:43.899441 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-b2b50605-b880-448e-87fd-85f52526aeba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:44.000085 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b50605-b880-448e-87fd-85f52526aeba None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:24:44.019934 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:44.073075 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cae535b-30f4-41fe-9501-0099917b1af1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:24:44.073685 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae535b-30f4-41fe-9501-0099917b1af1 service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:44.073972 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae535b-30f4-41fe-9501-0099917b1af1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:44.074245 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae535b-30f4-41fe-9501-0099917b1af1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:44.075849 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-4cae535b-30f4-41fe-9501-0099917b1af1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:44.183919 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cae535b-30f4-41fe-9501-0099917b1af1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:24:44.185398 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cae535b-30f4-41fe-9501-0099917b1af1 service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:24:44.203551 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 735/1452] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:24:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:24:46.315077 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7c954-ff73-4b39-9cd8-e496d06dfc9e None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:46.315605 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7c954-ff73-4b39-9cd8-e496d06dfc9e None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:46.315605 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7c954-ff73-4b39-9cd8-e496d06dfc9e None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:46.316931 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 718/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 15:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:24:46.322373 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709795a-d092-4232-9459-0ab6a9f1a701 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:46.322830 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709795a-d092-4232-9459-0ab6a9f1a701 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:46.322953 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709795a-d092-4232-9459-0ab6a9f1a701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:46.386560 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:46.389406 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-6709795a-d092-4232-9459-0ab6a9f1a701 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:24:46.390640 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-6709795a-d092-4232-9459-0ab6a9f1a701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:46.486267 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 736/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 15:24:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:24:46.524950 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f7c954-ff73-4b39-9cd8-e496d06dfc9e None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:24:46.526486 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-f6f7c954-ff73-4b39-9cd8-e496d06dfc9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:46.629227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f7c954-ff73-4b39-9cd8-e496d06dfc9e None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:24:46.677916 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:46.702284 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcca49a-9f84-4e7c-977f-bd07dea78ed8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:24:46.702895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcca49a-9f84-4e7c-977f-bd07dea78ed8 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:46.703179 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcca49a-9f84-4e7c-977f-bd07dea78ed8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:46.703455 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcca49a-9f84-4e7c-977f-bd07dea78ed8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:46.705109 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-edcca49a-9f84-4e7c-977f-bd07dea78ed8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:46.812675 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcca49a-9f84-4e7c-977f-bd07dea78ed8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:24:46.814119 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcca49a-9f84-4e7c-977f-bd07dea78ed8 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:24:46.832232 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 719/1455] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:24:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:24:48.890031 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f5e6a-b10e-4925-85ad-d8ee490dd504 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:48.890624 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f5e6a-b10e-4925-85ad-d8ee490dd504 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:48.890624 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f5e6a-b10e-4925-85ad-d8ee490dd504 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:48.891946 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 737/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 15:24:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:24:48.897191 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891307ec-967f-4384-976f-85e93f45803e None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:48.897675 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891307ec-967f-4384-976f-85e93f45803e None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:48.897790 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891307ec-967f-4384-976f-85e93f45803e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:48.964359 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-891307ec-967f-4384-976f-85e93f45803e None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:24:48.965581 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-891307ec-967f-4384-976f-85e93f45803e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:48.994176 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:49.061446 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 720/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 15:24:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:24:49.099745 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f5e6a-b10e-4925-85ad-d8ee490dd504 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:24:49.101932 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-785f5e6a-b10e-4925-85ad-d8ee490dd504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:49.210168 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f5e6a-b10e-4925-85ad-d8ee490dd504 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:24:49.238308 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:49.283772 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae9acc-3053-40e9-a7e9-121f7fc3f16f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:24:49.284449 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae9acc-3053-40e9-a7e9-121f7fc3f16f service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:49.284723 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae9acc-3053-40e9-a7e9-121f7fc3f16f service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:49.285003 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae9acc-3053-40e9-a7e9-121f7fc3f16f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:49.286674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-3fae9acc-3053-40e9-a7e9-121f7fc3f16f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:49.395540 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fae9acc-3053-40e9-a7e9-121f7fc3f16f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:24:49.396978 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fae9acc-3053-40e9-a7e9-121f7fc3f16f service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:24:49.415377 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 738/1458] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:24:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:24:51.512599 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e8592-565f-4f1c-a752-1be496374102 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:51.513142 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e8592-565f-4f1c-a752-1be496374102 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:51.513238 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e8592-565f-4f1c-a752-1be496374102 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:51.514535 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 721/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 15:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 15:24:51.519957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097d6c0-855f-4184-87ec-07f51f7fcc56 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:51.520400 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097d6c0-855f-4184-87ec-07f51f7fcc56 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:51.520551 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097d6c0-855f-4184-87ec-07f51f7fcc56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:51.586101 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-1097d6c0-855f-4184-87ec-07f51f7fcc56 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:24:51.587312 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-1097d6c0-855f-4184-87ec-07f51f7fcc56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:51.595491 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:51.684579 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 739/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 15:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:24:51.722314 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e8592-565f-4f1c-a752-1be496374102 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:24:51.723865 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-724e8592-565f-4f1c-a752-1be496374102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:51.826105 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e8592-565f-4f1c-a752-1be496374102 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:24:51.881100 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:51.900055 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893af787-db87-456a-9eb1-36b717fd4df6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:24:51.900695 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893af787-db87-456a-9eb1-36b717fd4df6 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:51.900980 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893af787-db87-456a-9eb1-36b717fd4df6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:51.901251 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893af787-db87-456a-9eb1-36b717fd4df6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:51.902911 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-893af787-db87-456a-9eb1-36b717fd4df6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:52.009715 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893af787-db87-456a-9eb1-36b717fd4df6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:24:52.011135 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893af787-db87-456a-9eb1-36b717fd4df6 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:24:52.029636 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 722/1461] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:24:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:24:54.088267 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92125e87-bae0-4d5e-a424-d8698fa84290 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:54.088840 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92125e87-bae0-4d5e-a424-d8698fa84290 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:54.088957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92125e87-bae0-4d5e-a424-d8698fa84290 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:54.090247 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 740/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 15:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:24:54.095729 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93918cc-b58e-4742-87f3-77a486652cf2 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:54.096186 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93918cc-b58e-4742-87f3-77a486652cf2 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:54.096335 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93918cc-b58e-4742-87f3-77a486652cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:54.161570 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-a93918cc-b58e-4742-87f3-77a486652cf2 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:24:54.162799 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-a93918cc-b58e-4742-87f3-77a486652cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:54.203549 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:54.262704 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 723/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 15:24:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:24:54.305228 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92125e87-bae0-4d5e-a424-d8698fa84290 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:24:54.306829 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-92125e87-bae0-4d5e-a424-d8698fa84290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:54.408811 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92125e87-bae0-4d5e-a424-d8698fa84290 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:24:54.447471 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:54.482692 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94618a3-5473-45f4-97aa-1115511fd72e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:24:54.483313 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94618a3-5473-45f4-97aa-1115511fd72e service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:54.483586 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94618a3-5473-45f4-97aa-1115511fd72e service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:54.483863 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94618a3-5473-45f4-97aa-1115511fd72e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:54.485522 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-a94618a3-5473-45f4-97aa-1115511fd72e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:54.592524 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94618a3-5473-45f4-97aa-1115511fd72e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:24:54.593978 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94618a3-5473-45f4-97aa-1115511fd72e service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:24:54.612339 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 741/1464] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:24:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:24:56.678623 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e7f21-3435-46ee-addd-3284983169d2 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:56.679158 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e7f21-3435-46ee-addd-3284983169d2 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:56.679255 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e7f21-3435-46ee-addd-3284983169d2 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:56.680615 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 724/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 15:24:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:24:56.685928 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9bcc9-8908-427f-b6af-de6164f002da None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:56.686340 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9bcc9-8908-427f-b6af-de6164f002da None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:56.686498 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9bcc9-8908-427f-b6af-de6164f002da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:56.751711 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-ebb9bcc9-8908-427f-b6af-de6164f002da None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:24:56.752932 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-ebb9bcc9-8908-427f-b6af-de6164f002da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:56.770748 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:56.847695 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 742/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 15:24:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:24:56.883924 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e7f21-3435-46ee-addd-3284983169d2 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:24:56.885510 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-532e7f21-3435-46ee-addd-3284983169d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:56.987105 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e7f21-3435-46ee-addd-3284983169d2 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:24:57.052445 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:57.059590 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a9e8ab-dee3-4b84-bea8-52ff358072f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:24:57.062203 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9e8ab-dee3-4b84-bea8-52ff358072f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:57.062203 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9e8ab-dee3-4b84-bea8-52ff358072f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:57.062203 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9e8ab-dee3-4b84-bea8-52ff358072f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:57.062553 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-51a9e8ab-dee3-4b84-bea8-52ff358072f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:57.169731 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a9e8ab-dee3-4b84-bea8-52ff358072f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:24:57.170968 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a9e8ab-dee3-4b84-bea8-52ff358072f7 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:24:57.189661 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 725/1467] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:24:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:24:59.218685 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf49ba5-8fbe-44a3-bc85-793f58809da8 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:59.219277 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf49ba5-8fbe-44a3-bc85-793f58809da8 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:59.219277 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf49ba5-8fbe-44a3-bc85-793f58809da8 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:59.220677 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 743/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 15:24:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:24:59.225937 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba856ab-7b3c-4d33-b1cc-2b3d0d1677f0 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:59.226383 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba856ab-7b3c-4d33-b1cc-2b3d0d1677f0 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:59.226489 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba856ab-7b3c-4d33-b1cc-2b3d0d1677f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:59.301050 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-2ba856ab-7b3c-4d33-b1cc-2b3d0d1677f0 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:24:59.302312 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-2ba856ab-7b3c-4d33-b1cc-2b3d0d1677f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:59.350225 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:59.398982 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 726/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 15:24:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:24:59.436107 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf49ba5-8fbe-44a3-bc85-793f58809da8 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:24:59.437651 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-1bf49ba5-8fbe-44a3-bc85-793f58809da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:59.541277 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf49ba5-8fbe-44a3-bc85-793f58809da8 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:24:59.587549 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:24:59.613790 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d87188-f45b-4279-bf51-900f24ffe458 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:24:59.614405 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d87188-f45b-4279-bf51-900f24ffe458 service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:24:59.614684 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d87188-f45b-4279-bf51-900f24ffe458 service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:24:59.614953 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d87188-f45b-4279-bf51-900f24ffe458 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:24:59.618042 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-96d87188-f45b-4279-bf51-900f24ffe458 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:24:59.725117 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d87188-f45b-4279-bf51-900f24ffe458 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:24:59.727028 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d87188-f45b-4279-bf51-900f24ffe458 service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:24:59.745327 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 744/1470] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:24:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 15:25:09.397070 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=22 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:25:09.743048 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=66 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:25:19.199122 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c1840-fdb1-4225-9d48-67ff78b4cc3b None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:25:19.199645 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c1840-fdb1-4225-9d48-67ff78b4cc3b None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:25:19.199754 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c1840-fdb1-4225-9d48-67ff78b4cc3b None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:25:19.201099 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 727/1471] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:25:19.206308 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3b65d-0fc8-4f4f-b0e6-3c6201aede7d None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:25:19.206738 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3b65d-0fc8-4f4f-b0e6-3c6201aede7d None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:25:19.206862 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3b65d-0fc8-4f4f-b0e6-3c6201aede7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:25:19.271887 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-4aa3b65d-0fc8-4f4f-b0e6-3c6201aede7d None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:25:19.273682 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-4aa3b65d-0fc8-4f4f-b0e6-3c6201aede7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:25:19.368122 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 745/1472] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:25:19.452045 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642e4c9-6bfa-430f-98c9-1d795bb4efcf None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:25:19.452355 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642e4c9-6bfa-430f-98c9-1d795bb4efcf None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:25:19.452631 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642e4c9-6bfa-430f-98c9-1d795bb4efcf None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:25:19.453967 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 728/1473] 10.222.0.40 () {54 vars in 966 bytes} [Fri Aug 5 15:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:25:19.458832 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d259ab-39a7-42be-993f-0867cc6407a3 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:25:19.459126 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d259ab-39a7-42be-993f-0867cc6407a3 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:25:19.459412 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d259ab-39a7-42be-993f-0867cc6407a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:25:19.536411 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-04d259ab-39a7-42be-993f-0867cc6407a3 None None] MFA Rules not processed for user `c386fa0483cf42efaddbc2395f6d1fce`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:25:19.537623 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-04d259ab-39a7-42be-993f-0867cc6407a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:25:19.558440 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:25:19.613542 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 746/1474] 10.222.0.40 () {58 vars in 1063 bytes} [Fri Aug 5 15:25:19 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:25:19.621085 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4642e4c9-6bfa-430f-98c9-1d795bb4efcf None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:25:19.622588 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-4642e4c9-6bfa-430f-98c9-1d795bb4efcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:25:19.728962 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4642e4c9-6bfa-430f-98c9-1d795bb4efcf None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:25:19.802502 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1813320f-abb1-4df5-b027-97296527565f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c386fa0483cf42efaddbc2395f6d1fce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:25:19.803118 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813320f-abb1-4df5-b027-97296527565f service glance] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:25:19.803944 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813320f-abb1-4df5-b027-97296527565f service glance] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:25:19.804252 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813320f-abb1-4df5-b027-97296527565f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:25:19.805885 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-1813320f-abb1-4df5-b027-97296527565f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:25:19.859677 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:25:19.915047 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1813320f-abb1-4df5-b027-97296527565f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:25:19.916513 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1813320f-abb1-4df5-b027-97296527565f service glance] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:25:19.935267 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 729/1475] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:25:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:25:22.039116 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416627aa-e17b-4a95-9193-541eda1bd66d None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:25:22.039698 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416627aa-e17b-4a95-9193-541eda1bd66d None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:25:22.039698 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416627aa-e17b-4a95-9193-541eda1bd66d None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:25:22.041120 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 747/1476] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:25:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:25:22.046626 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04df9bb-b7b1-40c1-adea-2111f6cb045e None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:25:22.047053 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04df9bb-b7b1-40c1-adea-2111f6cb045e None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:25:22.047215 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04df9bb-b7b1-40c1-adea-2111f6cb045e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:25:22.111667 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-c04df9bb-b7b1-40c1-adea-2111f6cb045e None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:25:22.112911 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-c04df9bb-b7b1-40c1-adea-2111f6cb045e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:25:22.211381 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 730/1477] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:25:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:25:22.234474 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416627aa-e17b-4a95-9193-541eda1bd66d None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:25:22.236045 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-416627aa-e17b-4a95-9193-541eda1bd66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:25:22.363962 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416627aa-e17b-4a95-9193-541eda1bd66d None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:25:22.415511 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:25:22.460509 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa7f28-bf69-4b4d-a6d0-c32bc2395c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:25:22.461122 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa7f28-bf69-4b4d-a6d0-c32bc2395c5a admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:25:22.461399 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa7f28-bf69-4b4d-a6d0-c32bc2395c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:25:22.461671 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa7f28-bf69-4b4d-a6d0-c32bc2395c5a admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:25:22.462357 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fa7f28-bf69-4b4d-a6d0-c32bc2395c5a admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:25:22.464106 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: WARNING py.warnings [None req-85fa7f28-bf69-4b4d-a6d0-c32bc2395c5a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 15:25:22.464106 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: warnings.warn(msg) Aug 05 15:25:22.464106 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: Aug 05 15:25:22.465246 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fa7f28-bf69-4b4d-a6d0-c32bc2395c5a admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:25:22.483376 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 748/1478] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 5 15:25:22 2022] GET /identity/v3/projects => generated 2553 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:25:24.498729 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa28db-6513-41b4-98ac-93b3baf4779e None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:25:24.499279 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa28db-6513-41b4-98ac-93b3baf4779e None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:25:24.499279 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa28db-6513-41b4-98ac-93b3baf4779e None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:25:24.500931 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 731/1479] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:25:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:25:24.506693 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d30b7f-9531-4c3c-a180-e49528a13f49 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:25:24.507124 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d30b7f-9531-4c3c-a180-e49528a13f49 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:25:24.507305 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d30b7f-9531-4c3c-a180-e49528a13f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:25:24.574956 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-66d30b7f-9531-4c3c-a180-e49528a13f49 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:25:24.576256 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-66d30b7f-9531-4c3c-a180-e49528a13f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:25:24.672899 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 749/1480] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:25:24.717846 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa28db-6513-41b4-98ac-93b3baf4779e None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:25:24.719472 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-73aa28db-6513-41b4-98ac-93b3baf4779e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:25:24.729910 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:25:24.867862 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa28db-6513-41b4-98ac-93b3baf4779e None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:25:24.947077 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:25:24.979818 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d92e3e2-90f4-45cc-8040-24c2069968bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:25:24.980490 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d92e3e2-90f4-45cc-8040-24c2069968bb service nova] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:25:24.980772 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d92e3e2-90f4-45cc-8040-24c2069968bb service nova] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:25:24.981045 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d92e3e2-90f4-45cc-8040-24c2069968bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:25:24.982723 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-9d92e3e2-90f4-45cc-8040-24c2069968bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:25:25.070407 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d92e3e2-90f4-45cc-8040-24c2069968bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:25:25.071973 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d92e3e2-90f4-45cc-8040-24c2069968bb service nova] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:25:25.089470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 732/1481] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:25:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:25:27.373435 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81cbb77-c5b3-42cf-bb51-434fb61415ca None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:25:27.373971 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81cbb77-c5b3-42cf-bb51-434fb61415ca None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:25:27.373971 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81cbb77-c5b3-42cf-bb51-434fb61415ca None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:25:27.375644 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 750/1482] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:25:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:25:27.382080 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dbc1c1-cfb9-4032-89f1-7835b096c94d None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:25:27.382614 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dbc1c1-cfb9-4032-89f1-7835b096c94d None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:25:27.382721 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dbc1c1-cfb9-4032-89f1-7835b096c94d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:25:27.453421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-74dbc1c1-cfb9-4032-89f1-7835b096c94d None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:25:27.454651 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-74dbc1c1-cfb9-4032-89f1-7835b096c94d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:25:27.500451 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:25:27.550676 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 733/1483] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:25:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:25:27.587416 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81cbb77-c5b3-42cf-bb51-434fb61415ca None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:25:27.589069 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-d81cbb77-c5b3-42cf-bb51-434fb61415ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:25:27.612495 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:25:27.700751 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81cbb77-c5b3-42cf-bb51-434fb61415ca None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:25:27.773806 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b2093f-19b4-4377-8ef6-19ce92795518 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:25:27.774422 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2093f-19b4-4377-8ef6-19ce92795518 service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:25:27.774709 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2093f-19b4-4377-8ef6-19ce92795518 service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:25:27.774987 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2093f-19b4-4377-8ef6-19ce92795518 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:25:27.776654 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-f2b2093f-19b4-4377-8ef6-19ce92795518 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:25:27.850791 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:25:27.885626 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b2093f-19b4-4377-8ef6-19ce92795518 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:25:27.887059 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b2093f-19b4-4377-8ef6-19ce92795518 service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:25:27.905592 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 751/1484] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:25:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:25:29.957827 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050d7e9-3e62-48e4-8102-b3463d282799 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:25:29.958418 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050d7e9-3e62-48e4-8102-b3463d282799 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:25:29.958418 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050d7e9-3e62-48e4-8102-b3463d282799 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:25:29.959780 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 734/1485] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 15:25:29.965158 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806624f-3597-4625-8076-3d848b4f6665 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:25:29.965572 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806624f-3597-4625-8076-3d848b4f6665 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:25:29.965726 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806624f-3597-4625-8076-3d848b4f6665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:25:30.030632 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-5806624f-3597-4625-8076-3d848b4f6665 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:25:30.031845 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-5806624f-3597-4625-8076-3d848b4f6665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:25:30.133202 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 752/1486] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:25:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:25:30.171145 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050d7e9-3e62-48e4-8102-b3463d282799 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:25:30.172747 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-0050d7e9-3e62-48e4-8102-b3463d282799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:25:30.275501 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050d7e9-3e62-48e4-8102-b3463d282799 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:25:30.347902 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:25:30.350315 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ef9ae-b047-48de-a2a0-c89b93aac5e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:25:30.350933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ef9ae-b047-48de-a2a0-c89b93aac5e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:25:30.351217 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ef9ae-b047-48de-a2a0-c89b93aac5e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:25:30.351495 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ef9ae-b047-48de-a2a0-c89b93aac5e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:25:30.353175 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-2e8ef9ae-b047-48de-a2a0-c89b93aac5e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:25:30.462443 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8ef9ae-b047-48de-a2a0-c89b93aac5e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:25:30.463823 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8ef9ae-b047-48de-a2a0-c89b93aac5e0 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:25:30.481999 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 735/1487] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:25:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:25:32.639710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44d90e-77df-4f34-ab94-721777c9f7c4 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:25:32.640307 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44d90e-77df-4f34-ab94-721777c9f7c4 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:25:32.640307 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44d90e-77df-4f34-ab94-721777c9f7c4 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:25:32.641664 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 753/1488] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:25:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:25:32.647371 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d91d1-10b8-463b-97b7-8bfc50998e7c None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:25:32.647811 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d91d1-10b8-463b-97b7-8bfc50998e7c None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:25:32.648015 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d91d1-10b8-463b-97b7-8bfc50998e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:25:32.713746 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-845d91d1-10b8-463b-97b7-8bfc50998e7c None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:25:32.714967 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-845d91d1-10b8-463b-97b7-8bfc50998e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:25:32.766905 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:25:32.810222 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 736/1489] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:25:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:25:32.850958 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d44d90e-77df-4f34-ab94-721777c9f7c4 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:25:32.852534 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-1d44d90e-77df-4f34-ab94-721777c9f7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:25:32.888945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:25:32.958006 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d44d90e-77df-4f34-ab94-721777c9f7c4 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:25:33.029757 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f633a8d-4092-4398-9f94-80ffadb26309 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:25:33.030371 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f633a8d-4092-4398-9f94-80ffadb26309 service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:25:33.030660 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f633a8d-4092-4398-9f94-80ffadb26309 service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:25:33.030934 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f633a8d-4092-4398-9f94-80ffadb26309 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:25:33.032588 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-6f633a8d-4092-4398-9f94-80ffadb26309 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:25:33.109733 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:25:33.140306 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f633a8d-4092-4398-9f94-80ffadb26309 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:25:33.141734 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f633a8d-4092-4398-9f94-80ffadb26309 service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:25:33.159757 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 754/1490] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:25:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:25:35.711444 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc4212-7ad7-4afb-99dd-4a3c8a86e640 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:25:35.712025 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc4212-7ad7-4afb-99dd-4a3c8a86e640 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:25:35.712025 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc4212-7ad7-4afb-99dd-4a3c8a86e640 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:25:35.713428 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 737/1491] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 5 15:25:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:25:35.718916 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b4a59-5a92-4a2b-89c9-fa4917b1e99c None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:25:35.719329 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b4a59-5a92-4a2b-89c9-fa4917b1e99c None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:25:35.719463 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b4a59-5a92-4a2b-89c9-fa4917b1e99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:25:35.785552 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-640b4a59-5a92-4a2b-89c9-fa4917b1e99c None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:25:35.786774 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-640b4a59-5a92-4a2b-89c9-fa4917b1e99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:25:35.890239 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 755/1492] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 5 15:25:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:25:35.930455 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc4212-7ad7-4afb-99dd-4a3c8a86e640 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:25:35.932076 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-0abc4212-7ad7-4afb-99dd-4a3c8a86e640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:25:36.033754 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc4212-7ad7-4afb-99dd-4a3c8a86e640 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:25:36.110599 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7264627a-c869-4f17-98f0-2cf63b50d0e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:25:36.111227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264627a-c869-4f17-98f0-2cf63b50d0e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:25:36.111514 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264627a-c869-4f17-98f0-2cf63b50d0e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:25:36.111794 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264627a-c869-4f17-98f0-2cf63b50d0e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:25:36.113518 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-7264627a-c869-4f17-98f0-2cf63b50d0e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:25:36.125744 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:25:36.228931 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7264627a-c869-4f17-98f0-2cf63b50d0e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:25:36.230347 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7264627a-c869-4f17-98f0-2cf63b50d0e5 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:25:36.248529 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 738/1493] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:25:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:25:45.888405 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=82 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:25:46.246196 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=50 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:27:21.247210 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6e007-bca9-4e89-bd12-173e31aa94bf None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:27:21.247975 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6e007-bca9-4e89-bd12-173e31aa94bf None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:27:21.248129 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6e007-bca9-4e89-bd12-173e31aa94bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:27:21.323428 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-05e6e007-bca9-4e89-bd12-173e31aa94bf None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:27:21.324683 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-05e6e007-bca9-4e89-bd12-173e31aa94bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:27:21.420001 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 756/1494] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:27:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:27:21.428999 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5664f-d0eb-4c47-9618-77919589c779 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:27:21.429508 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5664f-d0eb-4c47-9618-77919589c779 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:27:21.429613 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5664f-d0eb-4c47-9618-77919589c779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:27:21.504337 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-e0b5664f-d0eb-4c47-9618-77919589c779 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:27:21.505560 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-e0b5664f-d0eb-4c47-9618-77919589c779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:27:21.600009 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 739/1495] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:27:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:27:21.604601 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e6e007-bca9-4e89-bd12-173e31aa94bf None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:27:21.605796 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-05e6e007-bca9-4e89-bd12-173e31aa94bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:27:21.617882 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e6e007-bca9-4e89-bd12-173e31aa94bf None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:27:21.679336 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:27:21.712510 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dfc864-cbe4-49f3-9a48-dd376b10a6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:27:21.713135 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dfc864-cbe4-49f3-9a48-dd376b10a6d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:27:21.713418 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dfc864-cbe4-49f3-9a48-dd376b10a6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:27:21.713701 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dfc864-cbe4-49f3-9a48-dd376b10a6d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:27:21.714390 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dfc864-cbe4-49f3-9a48-dd376b10a6d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:27:21.714698 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dfc864-cbe4-49f3-9a48-dd376b10a6d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:27:21.715723 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: WARNING py.warnings [None req-55dfc864-cbe4-49f3-9a48-dd376b10a6d9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 05 15:27:21.715723 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: warnings.warn(msg) Aug 05 15:27:21.715723 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: Aug 05 15:27:21.716662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dfc864-cbe4-49f3-9a48-dd376b10a6d9 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:27:21.728945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 757/1496] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:27:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:27:21.737388 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5664f-d0eb-4c47-9618-77919589c779 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:27:21.746576 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5664f-d0eb-4c47-9618-77919589c779 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:27:21.808075 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:27:21.841536 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e098f68-e4fb-454c-8bb0-de8e2d95bfcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:27:21.842146 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e098f68-e4fb-454c-8bb0-de8e2d95bfcb admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:27:21.842425 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e098f68-e4fb-454c-8bb0-de8e2d95bfcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:27:21.842698 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e098f68-e4fb-454c-8bb0-de8e2d95bfcb admin admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:27:21.843347 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e098f68-e4fb-454c-8bb0-de8e2d95bfcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:27:21.845079 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: WARNING py.warnings [None req-2e098f68-e4fb-454c-8bb0-de8e2d95bfcb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 15:27:21.845079 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: warnings.warn(msg) Aug 05 15:27:21.845079 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: Aug 05 15:27:21.846916 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e098f68-e4fb-454c-8bb0-de8e2d95bfcb admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:27:21.962243 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 740/1497] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:27:21 2022] POST /identity/v3/projects => generated 377 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:27:21.966420 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dfc864-cbe4-49f3-9a48-dd376b10a6d9 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:27:21.967948 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-55dfc864-cbe4-49f3-9a48-dd376b10a6d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:27:22.001787 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dfc864-cbe4-49f3-9a48-dd376b10a6d9 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:27:22.097264 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc752ed-e1d4-4368-8b99-cd00ab6b9d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:27:22.097885 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc752ed-e1d4-4368-8b99-cd00ab6b9d47 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:27:22.098173 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc752ed-e1d4-4368-8b99-cd00ab6b9d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:27:22.098454 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc752ed-e1d4-4368-8b99-cd00ab6b9d47 admin admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:27:22.099126 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc752ed-e1d4-4368-8b99-cd00ab6b9d47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:27:22.100178 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: WARNING py.warnings [None req-ccc752ed-e1d4-4368-8b99-cd00ab6b9d47 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 15:27:22.100178 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: warnings.warn(msg) Aug 05 15:27:22.100178 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: Aug 05 15:27:22.101437 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc752ed-e1d4-4368-8b99-cd00ab6b9d47 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:27:22.217760 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 758/1498] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:27:21 2022] POST /identity/v3/users => generated 289 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:27:22.222013 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e098f68-e4fb-454c-8bb0-de8e2d95bfcb admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:27:22.223543 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-2e098f68-e4fb-454c-8bb0-de8e2d95bfcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:27:22.288000 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e098f68-e4fb-454c-8bb0-de8e2d95bfcb admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:27:22.374198 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:27:22.384869 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca248ead-8761-4daf-9244-e32fe29f94cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:27:22.385491 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca248ead-8761-4daf-9244-e32fe29f94cb admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:27:22.385778 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca248ead-8761-4daf-9244-e32fe29f94cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:27:22.386056 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca248ead-8761-4daf-9244-e32fe29f94cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:27:22.386748 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca248ead-8761-4daf-9244-e32fe29f94cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:27:22.387822 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: WARNING py.warnings [None req-ca248ead-8761-4daf-9244-e32fe29f94cb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 05 15:27:22.387822 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: warnings.warn(msg) Aug 05 15:27:22.387822 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: Aug 05 15:27:22.388817 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca248ead-8761-4daf-9244-e32fe29f94cb admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:27:22.401518 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 741/1499] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:27:22.405617 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc752ed-e1d4-4368-8b99-cd00ab6b9d47 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:27:22.407144 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-ccc752ed-e1d4-4368-8b99-cd00ab6b9d47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:27:22.419304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc752ed-e1d4-4368-8b99-cd00ab6b9d47 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:27:22.443747 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:27:22.511938 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ea735-3475-4f07-a663-69dd6578621b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:27:22.512589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ea735-3475-4f07-a663-69dd6578621b admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:27:22.512858 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ea735-3475-4f07-a663-69dd6578621b admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:27:22.513132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ea735-3475-4f07-a663-69dd6578621b admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:27:22.513819 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500ea735-3475-4f07-a663-69dd6578621b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:27:22.514823 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: WARNING py.warnings [None req-500ea735-3475-4f07-a663-69dd6578621b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 05 15:27:22.514823 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: warnings.warn(msg) Aug 05 15:27:22.514823 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: Aug 05 15:27:22.515706 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500ea735-3475-4f07-a663-69dd6578621b admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:27:22.528279 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 759/1500] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:27:22.532446 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca248ead-8761-4daf-9244-e32fe29f94cb admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:27:22.534754 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-ca248ead-8761-4daf-9244-e32fe29f94cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:27:22.568249 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca248ead-8761-4daf-9244-e32fe29f94cb admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:27:22.663850 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb88ab-9c98-452e-863a-5b722891ee55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:27:22.664552 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb88ab-9c98-452e-863a-5b722891ee55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:27:22.664835 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb88ab-9c98-452e-863a-5b722891ee55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:27:22.665113 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb88ab-9c98-452e-863a-5b722891ee55 admin admin] PATH_INFO: `/v3/projects/43c68167f2f9417f819006c29e1f6cee/users/c0478a1ddf674592b296a2090b090e1e/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:27:22.686531 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceb88ab-9c98-452e-863a-5b722891ee55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43c68167f2f9417f819006c29e1f6cee, user_id=c0478a1ddf674592b296a2090b090e1e, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:27:22.687549 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: WARNING py.warnings [None req-3ceb88ab-9c98-452e-863a-5b722891ee55 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 15:27:22.687549 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: warnings.warn(msg) Aug 05 15:27:22.687549 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: Aug 05 15:27:22.689588 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceb88ab-9c98-452e-863a-5b722891ee55 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:27:22.774881 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 742/1501] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:27:22 2022] PUT /identity/v3/projects/43c68167f2f9417f819006c29e1f6cee/users/c0478a1ddf674592b296a2090b090e1e/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:27:22.788195 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5d33a-e4aa-44b1-9cdb-f0e3a43e8eec None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:27:22.788493 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5d33a-e4aa-44b1-9cdb-f0e3a43e8eec None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:27:22.788815 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5d33a-e4aa-44b1-9cdb-f0e3a43e8eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:27:22.845264 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-e9e5d33a-e4aa-44b1-9cdb-f0e3a43e8eec None None] MFA Rules not processed for user `c0478a1ddf674592b296a2090b090e1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:27:22.846472 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-e9e5d33a-e4aa-44b1-9cdb-f0e3a43e8eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:27:22.928841 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 760/1502] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:27:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:27:22.943066 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610ba46-7653-421a-9060-86d1cc037d1b None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:27:22.943366 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610ba46-7653-421a-9060-86d1cc037d1b None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:27:22.943650 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610ba46-7653-421a-9060-86d1cc037d1b None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:27:22.945169 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 743/1503] 10.222.0.40 () {54 vars in 964 bytes} [Fri Aug 5 15:27:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:27:22.950050 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00881883-094d-4f9e-952a-00483ea74bfb None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:27:22.950350 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00881883-094d-4f9e-952a-00483ea74bfb None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:27:22.950631 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00881883-094d-4f9e-952a-00483ea74bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:27:22.982356 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:27:23.028390 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-00881883-094d-4f9e-952a-00483ea74bfb None None] MFA Rules not processed for user `e974f8d9a06d489884a3b00b95dd88e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:27:23.029524 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-00881883-094d-4f9e-952a-00483ea74bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:27:23.122137 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 761/1504] 10.222.0.40 () {58 vars in 1061 bytes} [Fri Aug 5 15:27:22 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:27:23.130823 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a610ba46-7653-421a-9060-86d1cc037d1b None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:27:23.132371 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-a610ba46-7653-421a-9060-86d1cc037d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:27:23.183140 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:27:23.237574 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a610ba46-7653-421a-9060-86d1cc037d1b None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:27:23.314470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df33d70-d9bd-412b-828d-f9194b7c37d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:27:23.315581 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df33d70-d9bd-412b-828d-f9194b7c37d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:27:23.315866 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df33d70-d9bd-412b-828d-f9194b7c37d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:27:23.316176 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df33d70-d9bd-412b-828d-f9194b7c37d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:27:23.317814 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-4df33d70-d9bd-412b-828d-f9194b7c37d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:27:23.413537 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df33d70-d9bd-412b-828d-f9194b7c37d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:27:23.414967 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df33d70-d9bd-412b-828d-f9194b7c37d1 service cinder] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:27:23.418438 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:27:23.434492 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 744/1505] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:27:23 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:27:23.479374 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d040045-ad52-4afb-b677-f2d07975d85e None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:27:23.479825 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d040045-ad52-4afb-b677-f2d07975d85e None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:27:23.479932 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d040045-ad52-4afb-b677-f2d07975d85e None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:27:23.481206 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 762/1506] 10.222.0.40 () {52 vars in 830 bytes} [Fri Aug 5 15:27:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:27:23.484751 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df33d70-d9bd-412b-828d-f9194b7c37d1 service cinder] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:27:23.485962 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-4df33d70-d9bd-412b-828d-f9194b7c37d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:27:23.592265 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df33d70-d9bd-412b-828d-f9194b7c37d1 service cinder] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:27:23.698761 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae992c2-6eb5-4be4-b45d-c11e14aa77e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:27:23.699387 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae992c2-6eb5-4be4-b45d-c11e14aa77e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:27:23.699676 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae992c2-6eb5-4be4-b45d-c11e14aa77e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:27:23.699945 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae992c2-6eb5-4be4-b45d-c11e14aa77e9 admin admin] PATH_INFO: `/v3/users/c0478a1ddf674592b296a2090b090e1e` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:27:23.711788 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae992c2-6eb5-4be4-b45d-c11e14aa77e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0478a1ddf674592b296a2090b090e1e)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:27:23.712838 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: WARNING py.warnings [None req-bae992c2-6eb5-4be4-b45d-c11e14aa77e9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 15:27:23.712838 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: warnings.warn(msg) Aug 05 15:27:23.712838 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: Aug 05 15:27:23.714040 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae992c2-6eb5-4be4-b45d-c11e14aa77e9 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:27:23.724252 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:27:23.876326 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-bae992c2-6eb5-4be4-b45d-c11e14aa77e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0478a1ddf674592b296a2090b090e1e'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:27:23.890283 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-bae992c2-6eb5-4be4-b45d-c11e14aa77e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0478a1ddf674592b296a2090b090e1e'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:27:23.890735 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-bae992c2-6eb5-4be4-b45d-c11e14aa77e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0478a1ddf674592b296a2090b090e1e'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:27:23.913952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-bae992c2-6eb5-4be4-b45d-c11e14aa77e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0478a1ddf674592b296a2090b090e1e'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:27:23.914528 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-bae992c2-6eb5-4be4-b45d-c11e14aa77e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0478a1ddf674592b296a2090b090e1e'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:27:23.972356 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 745/1507] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:27:23 2022] DELETE /identity/v3/users/c0478a1ddf674592b296a2090b090e1e => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:27:23.976780 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32007580-dddb-46c5-9efd-74741c59de0f None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:27:23.977088 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32007580-dddb-46c5-9efd-74741c59de0f None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:27:23.977373 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32007580-dddb-46c5-9efd-74741c59de0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:27:24.033074 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-32007580-dddb-46c5-9efd-74741c59de0f None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:27:24.034886 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-32007580-dddb-46c5-9efd-74741c59de0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:27:24.110132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:27:24.123149 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 763/1508] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:27:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:27:24.132708 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae992c2-6eb5-4be4-b45d-c11e14aa77e9 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:27:24.133974 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-bae992c2-6eb5-4be4-b45d-c11e14aa77e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:27:24.238098 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae992c2-6eb5-4be4-b45d-c11e14aa77e9 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:27:24.313843 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:27:24.333539 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d0948-7d5a-4b32-a0e9-9c842c1516a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:27:24.334194 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d0948-7d5a-4b32-a0e9-9c842c1516a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:27:24.334478 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d0948-7d5a-4b32-a0e9-9c842c1516a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:27:24.334821 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d0948-7d5a-4b32-a0e9-9c842c1516a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:27:24.336449 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-556d0948-7d5a-4b32-a0e9-9c842c1516a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:27:24.444385 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556d0948-7d5a-4b32-a0e9-9c842c1516a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:27:24.445732 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556d0948-7d5a-4b32-a0e9-9c842c1516a1 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:27:24.464394 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 746/1509] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:27:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:27:24.924297 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32007580-dddb-46c5-9efd-74741c59de0f None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:27:24.925987 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-32007580-dddb-46c5-9efd-74741c59de0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:27:24.938448 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32007580-dddb-46c5-9efd-74741c59de0f None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:27:25.042305 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261fa4c4-158c-4e51-ab96-04b10f560fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:27:25.042956 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261fa4c4-158c-4e51-ab96-04b10f560fcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:27:25.043227 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261fa4c4-158c-4e51-ab96-04b10f560fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:27:25.043945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261fa4c4-158c-4e51-ab96-04b10f560fcd admin admin] PATH_INFO: `/v3/projects/43c68167f2f9417f819006c29e1f6cee` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:27:25.054814 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261fa4c4-158c-4e51-ab96-04b10f560fcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=43c68167f2f9417f819006c29e1f6cee)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:27:25.056046 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: WARNING py.warnings [None req-261fa4c4-158c-4e51-ab96-04b10f560fcd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 15:27:25.056046 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: warnings.warn(msg) Aug 05 15:27:25.056046 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: Aug 05 15:27:25.057274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261fa4c4-158c-4e51-ab96-04b10f560fcd admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:27:25.170998 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-261fa4c4-158c-4e51-ab96-04b10f560fcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43c68167f2f9417f819006c29e1f6cee', 'request_id': 'req-261fa4c4-158c-4e51-ab96-04b10f560fcd'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:27:25.185019 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-261fa4c4-158c-4e51-ab96-04b10f560fcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43c68167f2f9417f819006c29e1f6cee', 'request_id': 'req-261fa4c4-158c-4e51-ab96-04b10f560fcd'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:27:25.191098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-261fa4c4-158c-4e51-ab96-04b10f560fcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43c68167f2f9417f819006c29e1f6cee', 'request_id': 'req-261fa4c4-158c-4e51-ab96-04b10f560fcd'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:27:25.308607 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-261fa4c4-158c-4e51-ab96-04b10f560fcd admin admin] The token cache is being invalidate because project 43c68167f2f9417f819006c29e1f6cee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:27:25.308899 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-261fa4c4-158c-4e51-ab96-04b10f560fcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:27:25.309419 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-261fa4c4-158c-4e51-ab96-04b10f560fcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:27:25.310748 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 764/1510] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:27:24 2022] DELETE /identity/v3/projects/43c68167f2f9417f819006c29e1f6cee => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:27:34.462278 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=62 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:27:35.306549 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:28:53.665870 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3044b-7f0b-46f6-ae00-228dca6c5241 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:28:53.666492 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3044b-7f0b-46f6-ae00-228dca6c5241 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:28:53.666492 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3044b-7f0b-46f6-ae00-228dca6c5241 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:28:53.668435 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 747/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 15:28:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:28:53.673363 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64c35d-fc70-4967-b83f-de97be9cd388 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:28:53.673830 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64c35d-fc70-4967-b83f-de97be9cd388 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:28:53.673943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64c35d-fc70-4967-b83f-de97be9cd388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:28:53.756455 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-2d64c35d-fc70-4967-b83f-de97be9cd388 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:28:53.757681 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-2d64c35d-fc70-4967-b83f-de97be9cd388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:28:53.860706 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 765/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 15:28:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:28:54.013197 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3044b-7f0b-46f6-ae00-228dca6c5241 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:28:54.015324 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-2bb3044b-7f0b-46f6-ae00-228dca6c5241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:28:54.132208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3044b-7f0b-46f6-ae00-228dca6c5241 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:28:54.138282 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c460fa-6968-42c9-b662-d67d4a6622c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:28:54.138898 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c460fa-6968-42c9-b662-d67d4a6622c3 service nova] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:28:54.139181 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c460fa-6968-42c9-b662-d67d4a6622c3 service nova] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:28:54.139489 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c460fa-6968-42c9-b662-d67d4a6622c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:28:54.147506 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c460fa-6968-42c9-b662-d67d4a6622c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:28:54.149113 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c460fa-6968-42c9-b662-d67d4a6622c3 service nova] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:28:54.159005 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 748/1513] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:28:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:28:54.989990 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343eff1f-8dbc-423d-81fa-2cfa7c2288b7 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:28:54.990564 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343eff1f-8dbc-423d-81fa-2cfa7c2288b7 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:28:54.990564 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343eff1f-8dbc-423d-81fa-2cfa7c2288b7 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:28:54.991935 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 766/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 15:28:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:28:54.997472 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7e4dd-6e42-4d5d-a788-a24ff87d5187 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:28:54.997929 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7e4dd-6e42-4d5d-a788-a24ff87d5187 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:28:54.998054 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7e4dd-6e42-4d5d-a788-a24ff87d5187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:28:55.017269 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-6fb7e4dd-6e42-4d5d-a788-a24ff87d5187 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:28:55.018459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-6fb7e4dd-6e42-4d5d-a788-a24ff87d5187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:28:55.026304 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 749/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 15:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:28:55.183575 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343eff1f-8dbc-423d-81fa-2cfa7c2288b7 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:28:55.197941 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343eff1f-8dbc-423d-81fa-2cfa7c2288b7 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:28:55.203417 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde39c62-6d6d-48c7-a4df-d159c0fff187 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:28:55.204029 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde39c62-6d6d-48c7-a4df-d159c0fff187 service nova] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:28:55.204343 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde39c62-6d6d-48c7-a4df-d159c0fff187 service nova] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:28:55.204631 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde39c62-6d6d-48c7-a4df-d159c0fff187 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:28:55.214525 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde39c62-6d6d-48c7-a4df-d159c0fff187 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:28:55.216192 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde39c62-6d6d-48c7-a4df-d159c0fff187 service nova] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:28:55.226063 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 767/1516] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:28:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:28:55.949947 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1980d7-6c4e-402d-8171-d61a9c525673 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:28:55.950234 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1980d7-6c4e-402d-8171-d61a9c525673 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:28:55.950517 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1980d7-6c4e-402d-8171-d61a9c525673 None None] PATH_INFO: `/v3` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:28:55.951810 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 750/1517] 10.222.0.17 () {54 vars in 925 bytes} [Fri Aug 5 15:28:55 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:28:55.957928 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950530a-ea51-4902-a241-57508b21b00a None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:28:55.958226 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950530a-ea51-4902-a241-57508b21b00a None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:28:55.958508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950530a-ea51-4902-a241-57508b21b00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:28:56.017312 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-1950530a-ea51-4902-a241-57508b21b00a None None] MFA Rules not processed for user `107a7c18a2c6440b86e1b7c3e39cc724`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:28:56.018521 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-1950530a-ea51-4902-a241-57508b21b00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:28:56.035063 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:28:56.089640 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 768/1518] 10.222.0.17 () {58 vars in 1015 bytes} [Fri Aug 5 15:28:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:29:00.129644 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e225c6-e7c6-4ebb-ac59-3492ce649f59 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:29:00.130204 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e225c6-e7c6-4ebb-ac59-3492ce649f59 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:29:00.130204 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e225c6-e7c6-4ebb-ac59-3492ce649f59 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:29:00.131470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 751/1519] 10.222.0.40 () {54 vars in 961 bytes} [Fri Aug 5 15:29:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:29:00.136559 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17dcd3-3745-40dc-a565-d5e080f1d848 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:29:00.136976 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17dcd3-3745-40dc-a565-d5e080f1d848 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:29:00.137121 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17dcd3-3745-40dc-a565-d5e080f1d848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:29:00.156121 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-ad17dcd3-3745-40dc-a565-d5e080f1d848 None None] MFA Rules not processed for user `107a7c18a2c6440b86e1b7c3e39cc724`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:29:00.157318 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-ad17dcd3-3745-40dc-a565-d5e080f1d848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:29:00.165976 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 769/1520] 10.222.0.40 () {58 vars in 1058 bytes} [Fri Aug 5 15:29:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:29:00.174028 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e225c6-e7c6-4ebb-ac59-3492ce649f59 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:29:00.187758 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e225c6-e7c6-4ebb-ac59-3492ce649f59 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:29:00.194440 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b79caa-54ff-4f81-9fbf-33c79497659d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '107a7c18a2c6440b86e1b7c3e39cc724', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:29:00.195052 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b79caa-54ff-4f81-9fbf-33c79497659d service placement] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:29:00.195334 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b79caa-54ff-4f81-9fbf-33c79497659d service placement] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:29:00.195610 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b79caa-54ff-4f81-9fbf-33c79497659d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:29:00.204028 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b79caa-54ff-4f81-9fbf-33c79497659d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:29:00.205743 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b79caa-54ff-4f81-9fbf-33c79497659d service placement] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:29:00.215284 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 752/1521] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:29:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 15:29:06.451050 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a807d-075c-4762-ba55-aed0a0f9b67c None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:29:06.451577 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a807d-075c-4762-ba55-aed0a0f9b67c None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:29:06.451577 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a807d-075c-4762-ba55-aed0a0f9b67c None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:29:06.452909 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 770/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 15:29:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:29:06.457718 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f83fc-eb3b-4f0b-86e9-26030666afcf None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:29:06.458192 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f83fc-eb3b-4f0b-86e9-26030666afcf None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:29:06.458301 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f83fc-eb3b-4f0b-86e9-26030666afcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:29:06.478332 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-4e3f83fc-eb3b-4f0b-86e9-26030666afcf None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:29:06.480191 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-4e3f83fc-eb3b-4f0b-86e9-26030666afcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:29:06.488238 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 753/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 15:29:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:29:06.644785 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a807d-075c-4762-ba55-aed0a0f9b67c None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:29:06.659836 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a807d-075c-4762-ba55-aed0a0f9b67c None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:29:06.665245 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f9295-1e8b-49a7-97fc-1536494beacc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:29:06.665846 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f9295-1e8b-49a7-97fc-1536494beacc service nova] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:29:06.666129 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f9295-1e8b-49a7-97fc-1536494beacc service nova] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:29:06.666428 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f9295-1e8b-49a7-97fc-1536494beacc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:29:06.674877 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93f9295-1e8b-49a7-97fc-1536494beacc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:29:06.676497 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93f9295-1e8b-49a7-97fc-1536494beacc service nova] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:29:06.685896 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 771/1524] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:29:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:29:07.513874 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67960c2a-28e9-44d9-b45d-6ca690a4a780 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:29:07.514435 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67960c2a-28e9-44d9-b45d-6ca690a4a780 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:29:07.514435 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67960c2a-28e9-44d9-b45d-6ca690a4a780 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:29:07.515779 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 754/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 15:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:29:07.520546 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243a3be-0b48-47e2-9d11-679ea2f0fdf9 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:29:07.520946 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243a3be-0b48-47e2-9d11-679ea2f0fdf9 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:29:07.521142 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243a3be-0b48-47e2-9d11-679ea2f0fdf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:29:07.540201 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-b243a3be-0b48-47e2-9d11-679ea2f0fdf9 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:29:07.541911 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-b243a3be-0b48-47e2-9d11-679ea2f0fdf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:29:07.549741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 772/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 15:29:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:29:07.699362 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67960c2a-28e9-44d9-b45d-6ca690a4a780 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:29:07.713351 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67960c2a-28e9-44d9-b45d-6ca690a4a780 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:29:07.718789 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed7675-9438-4bfd-bee2-0f1b493782de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:29:07.719390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed7675-9438-4bfd-bee2-0f1b493782de service nova] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:29:07.719671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed7675-9438-4bfd-bee2-0f1b493782de service nova] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:29:07.719949 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed7675-9438-4bfd-bee2-0f1b493782de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:29:07.727865 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed7675-9438-4bfd-bee2-0f1b493782de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:29:07.730013 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed7675-9438-4bfd-bee2-0f1b493782de service nova] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:29:07.739296 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 755/1527] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:29:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:29:11.640739 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c92e2-0af6-490d-b2a6-c5aadb7e090d None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:29:11.641386 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c92e2-0af6-490d-b2a6-c5aadb7e090d None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:29:11.641386 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c92e2-0af6-490d-b2a6-c5aadb7e090d None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:29:11.642673 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 773/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 15:29:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:29:11.647709 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be419cbe-7659-4897-ab25-a005c3c59cc1 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:29:11.648171 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be419cbe-7659-4897-ab25-a005c3c59cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:29:11.648341 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be419cbe-7659-4897-ab25-a005c3c59cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:29:11.668606 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-be419cbe-7659-4897-ab25-a005c3c59cc1 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:29:11.669808 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-be419cbe-7659-4897-ab25-a005c3c59cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:29:11.677598 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 756/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 15:29:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:29:11.749941 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c92e2-0af6-490d-b2a6-c5aadb7e090d None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:29:11.751665 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-517c92e2-0af6-490d-b2a6-c5aadb7e090d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:29:11.829877 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c92e2-0af6-490d-b2a6-c5aadb7e090d None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:29:11.833985 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12656db-1c8a-4f24-84d6-c4905962912f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:29:11.834644 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12656db-1c8a-4f24-84d6-c4905962912f service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:29:11.834935 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12656db-1c8a-4f24-84d6-c4905962912f service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:29:11.835216 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12656db-1c8a-4f24-84d6-c4905962912f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:29:11.844616 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12656db-1c8a-4f24-84d6-c4905962912f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:29:11.846059 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12656db-1c8a-4f24-84d6-c4905962912f service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:29:11.855655 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 774/1530] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:29:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:29:13.860993 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65621e7-7776-41fc-9fa6-b99bde5f3ba1 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:29:13.861540 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65621e7-7776-41fc-9fa6-b99bde5f3ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:29:13.861669 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65621e7-7776-41fc-9fa6-b99bde5f3ba1 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:29:13.862942 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 757/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 15:29:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:29:13.867971 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83777d3a-59fc-4ad8-ba84-98b0e77e1fdc None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:29:13.868465 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83777d3a-59fc-4ad8-ba84-98b0e77e1fdc None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:29:13.868565 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83777d3a-59fc-4ad8-ba84-98b0e77e1fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:29:13.888757 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-83777d3a-59fc-4ad8-ba84-98b0e77e1fdc None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:29:13.889943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-83777d3a-59fc-4ad8-ba84-98b0e77e1fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:29:13.897639 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 775/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 15:29:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:29:13.934351 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65621e7-7776-41fc-9fa6-b99bde5f3ba1 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:29:13.936009 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-a65621e7-7776-41fc-9fa6-b99bde5f3ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:29:13.948897 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65621e7-7776-41fc-9fa6-b99bde5f3ba1 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:29:13.952425 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c1d1e-2bd6-43f8-87c6-d6290165ac9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:29:13.953027 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c1d1e-2bd6-43f8-87c6-d6290165ac9c service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:29:13.953312 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c1d1e-2bd6-43f8-87c6-d6290165ac9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:29:13.953592 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c1d1e-2bd6-43f8-87c6-d6290165ac9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:29:13.962437 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308c1d1e-2bd6-43f8-87c6-d6290165ac9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:29:13.964402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308c1d1e-2bd6-43f8-87c6-d6290165ac9c service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:29:13.977459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 758/1533] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:29:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:29:17.388029 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6d474-975a-4e99-a608-0d936739c41c None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:29:17.388614 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6d474-975a-4e99-a608-0d936739c41c None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:29:17.388614 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6d474-975a-4e99-a608-0d936739c41c None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:29:17.390041 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 776/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 15:29:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:29:17.395343 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e245f-ff91-47da-88f9-d86c8575396a None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:29:17.395771 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e245f-ff91-47da-88f9-d86c8575396a None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:29:17.395887 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e245f-ff91-47da-88f9-d86c8575396a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:29:17.404210 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=81 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:29:17.416164 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-d55e245f-ff91-47da-88f9-d86c8575396a None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:29:17.417387 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-d55e245f-ff91-47da-88f9-d86c8575396a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:29:17.425451 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 759/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 15:29:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:29:17.727863 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b6d474-975a-4e99-a608-0d936739c41c None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:29:17.729605 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-89b6d474-975a-4e99-a608-0d936739c41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:29:17.794366 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:29:17.807804 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b6d474-975a-4e99-a608-0d936739c41c None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:29:17.811488 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d0c74e-5b88-4dca-95cf-daddf903cae2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:29:17.812128 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d0c74e-5b88-4dca-95cf-daddf903cae2 service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:29:17.812414 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d0c74e-5b88-4dca-95cf-daddf903cae2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:29:17.812693 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d0c74e-5b88-4dca-95cf-daddf903cae2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:29:17.822453 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d0c74e-5b88-4dca-95cf-daddf903cae2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:29:17.823877 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d0c74e-5b88-4dca-95cf-daddf903cae2 service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:29:17.834049 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 777/1536] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:29:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:29:19.893574 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a012d-eb21-4828-b0e0-2b772c0917a5 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:29:19.894117 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a012d-eb21-4828-b0e0-2b772c0917a5 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:29:19.894117 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a012d-eb21-4828-b0e0-2b772c0917a5 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:29:19.895472 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 760/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 15:29:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:29:19.901190 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d6e4a-efbd-4c46-87a9-dfa07a5fe3e8 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:29:19.901593 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d6e4a-efbd-4c46-87a9-dfa07a5fe3e8 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:29:19.901754 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d6e4a-efbd-4c46-87a9-dfa07a5fe3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:29:19.921588 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-5a1d6e4a-efbd-4c46-87a9-dfa07a5fe3e8 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:29:19.922789 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-5a1d6e4a-efbd-4c46-87a9-dfa07a5fe3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:29:19.930706 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 778/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 15:29:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:29:20.232202 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a012d-eb21-4828-b0e0-2b772c0917a5 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:29:20.233929 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-c50a012d-eb21-4828-b0e0-2b772c0917a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:29:20.246994 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a012d-eb21-4828-b0e0-2b772c0917a5 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:29:20.250483 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac6c8d9-b128-47c8-a460-ada95007e22b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:29:20.251080 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac6c8d9-b128-47c8-a460-ada95007e22b service cinder] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:29:20.251363 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac6c8d9-b128-47c8-a460-ada95007e22b service cinder] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:29:20.251647 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac6c8d9-b128-47c8-a460-ada95007e22b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:29:20.260977 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac6c8d9-b128-47c8-a460-ada95007e22b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:29:20.262382 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac6c8d9-b128-47c8-a460-ada95007e22b service cinder] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:29:20.271498 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 761/1539] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:29:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:29:29.916813 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=11 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:29:30.267495 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=8 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:30:44.111999 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca35abb-608d-4020-86f3-5426b705f953 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:44.114898 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca35abb-608d-4020-86f3-5426b705f953 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:44.114898 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca35abb-608d-4020-86f3-5426b705f953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:44.136141 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-8ca35abb-608d-4020-86f3-5426b705f953 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:44.137363 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-8ca35abb-608d-4020-86f3-5426b705f953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:44.153237 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 779/1540] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:30:44.177135 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1468f4f-420a-4086-9e85-d7004beac305 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:44.177686 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1468f4f-420a-4086-9e85-d7004beac305 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:44.177686 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1468f4f-420a-4086-9e85-d7004beac305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:44.186091 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a46bb-7f2e-4f54-9f47-206dfdf74f75 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:44.186521 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a46bb-7f2e-4f54-9f47-206dfdf74f75 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:44.186654 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a46bb-7f2e-4f54-9f47-206dfdf74f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:44.213356 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-a1468f4f-420a-4086-9e85-d7004beac305 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:44.215018 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-a1468f4f-420a-4086-9e85-d7004beac305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:44.218507 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-944a46bb-7f2e-4f54-9f47-206dfdf74f75 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:44.219710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-944a46bb-7f2e-4f54-9f47-206dfdf74f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:44.232529 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 762/1541] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:30:44.234315 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 780/1542] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:30:44.239609 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1468f4f-420a-4086-9e85-d7004beac305 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:44.244477 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871e771-f499-48e1-90a3-429b19c14f56 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:44.244670 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871e771-f499-48e1-90a3-429b19c14f56 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:44.244946 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871e771-f499-48e1-90a3-429b19c14f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:44.252296 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1468f4f-420a-4086-9e85-d7004beac305 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:44.262374 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2599e260-2c27-42eb-85e5-07feab8eb8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:44.263030 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2599e260-2c27-42eb-85e5-07feab8eb8db admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:44.263315 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2599e260-2c27-42eb-85e5-07feab8eb8db admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:44.263594 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2599e260-2c27-42eb-85e5-07feab8eb8db admin admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:44.264339 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2599e260-2c27-42eb-85e5-07feab8eb8db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:30:44.264653 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2599e260-2c27-42eb-85e5-07feab8eb8db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:44.265700 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: WARNING py.warnings [None req-2599e260-2c27-42eb-85e5-07feab8eb8db admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 05 15:30:44.265700 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: warnings.warn(msg) Aug 05 15:30:44.265700 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: Aug 05 15:30:44.266974 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2599e260-2c27-42eb-85e5-07feab8eb8db admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:44.274114 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-e871e771-f499-48e1-90a3-429b19c14f56 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:44.275364 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-e871e771-f499-48e1-90a3-429b19c14f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:44.289088 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 763/1543] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:30:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:30:44.291062 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6446ff-a264-4cda-a323-c3511ceaae0b None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:44.291360 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6446ff-a264-4cda-a323-c3511ceaae0b None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:44.291640 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6446ff-a264-4cda-a323-c3511ceaae0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:44.294828 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 781/1544] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:30:44.296203 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f8d0b-d0cf-4ae9-a4b4-f624f11921b1 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:44.296492 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f8d0b-d0cf-4ae9-a4b4-f624f11921b1 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:44.296775 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f8d0b-d0cf-4ae9-a4b4-f624f11921b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:44.326078 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-bf6446ff-a264-4cda-a323-c3511ceaae0b None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:44.327333 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-bf6446ff-a264-4cda-a323-c3511ceaae0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:44.335832 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-3f6f8d0b-d0cf-4ae9-a4b4-f624f11921b1 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:44.337091 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-3f6f8d0b-d0cf-4ae9-a4b4-f624f11921b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:44.345859 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 764/1545] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:30:44.346478 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6446ff-a264-4cda-a323-c3511ceaae0b None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:44.356770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 782/1546] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:30:44.357880 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c63fc4e-6b88-4fbe-ba3c-8e14b70fbe94 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:44.358171 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c63fc4e-6b88-4fbe-ba3c-8e14b70fbe94 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:44.358451 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c63fc4e-6b88-4fbe-ba3c-8e14b70fbe94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:44.366601 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6446ff-a264-4cda-a323-c3511ceaae0b None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:44.376751 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca52ef-eed9-463d-be6c-146f3f0e23e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:44.377381 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca52ef-eed9-463d-be6c-146f3f0e23e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:44.377662 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca52ef-eed9-463d-be6c-146f3f0e23e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:44.377938 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca52ef-eed9-463d-be6c-146f3f0e23e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:44.378646 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca52ef-eed9-463d-be6c-146f3f0e23e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:30:44.378951 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca52ef-eed9-463d-be6c-146f3f0e23e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:44.380623 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca52ef-eed9-463d-be6c-146f3f0e23e0 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:44.397805 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-7c63fc4e-6b88-4fbe-ba3c-8e14b70fbe94 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:44.399096 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-7c63fc4e-6b88-4fbe-ba3c-8e14b70fbe94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:44.407033 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 765/1547] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:30:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:30:44.407697 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca52ef-eed9-463d-be6c-146f3f0e23e0 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:44.417546 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 783/1548] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:30:44.418652 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d3a62-961a-4ae5-aa34-5262b3b1a653 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:44.418942 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d3a62-961a-4ae5-aa34-5262b3b1a653 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:44.419220 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d3a62-961a-4ae5-aa34-5262b3b1a653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:44.429155 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca52ef-eed9-463d-be6c-146f3f0e23e0 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:44.439756 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3095da-2f6a-43fe-a6b0-81c9b1f8ee1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:44.440824 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3095da-2f6a-43fe-a6b0-81c9b1f8ee1c admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:44.440824 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3095da-2f6a-43fe-a6b0-81c9b1f8ee1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:44.440998 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3095da-2f6a-43fe-a6b0-81c9b1f8ee1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:44.441608 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3095da-2f6a-43fe-a6b0-81c9b1f8ee1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:44.443553 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3095da-2f6a-43fe-a6b0-81c9b1f8ee1c admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:44.458897 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-b04d3a62-961a-4ae5-aa34-5262b3b1a653 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:44.461333 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-b04d3a62-961a-4ae5-aa34-5262b3b1a653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:44.479760 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 784/1549] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:30:44.481266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e0d30-c42c-48bd-b766-5ad51a59178a None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:44.481554 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e0d30-c42c-48bd-b766-5ad51a59178a None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:44.481831 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e0d30-c42c-48bd-b766-5ad51a59178a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:44.504772 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-187e0d30-c42c-48bd-b766-5ad51a59178a None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:44.506259 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-187e0d30-c42c-48bd-b766-5ad51a59178a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:44.514995 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 785/1550] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:30:44.516083 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187e0d30-c42c-48bd-b766-5ad51a59178a None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:44.526355 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187e0d30-c42c-48bd-b766-5ad51a59178a None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:44.531529 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff649a-ac44-4f51-bef4-16c3be506410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:44.536361 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff649a-ac44-4f51-bef4-16c3be506410 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:44.536361 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff649a-ac44-4f51-bef4-16c3be506410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:44.536361 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff649a-ac44-4f51-bef4-16c3be506410 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:44.536361 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faff649a-ac44-4f51-bef4-16c3be506410 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:44.536361 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: WARNING py.warnings [None req-faff649a-ac44-4f51-bef4-16c3be506410 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 15:30:44.536361 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: warnings.warn(msg) Aug 05 15:30:44.536361 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: Aug 05 15:30:44.536936 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faff649a-ac44-4f51-bef4-16c3be506410 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:44.563447 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 766/1551] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:30:44 2022] POST /identity/v3/projects => generated 379 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:30:44.566188 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac45f9-4a3d-4adf-b798-c02eeb28c62d None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:44.566188 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac45f9-4a3d-4adf-b798-c02eeb28c62d None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:44.566188 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac45f9-4a3d-4adf-b798-c02eeb28c62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:44.584647 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-b1ac45f9-4a3d-4adf-b798-c02eeb28c62d None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:44.585836 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-b1ac45f9-4a3d-4adf-b798-c02eeb28c62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:44.629808 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 786/1552] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:30:44 2022] POST /identity/v3/projects => generated 393 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:30:44.629808 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a54cf-0e86-4b0d-bb5c-b2f3cd7e26dd None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:44.629808 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a54cf-0e86-4b0d-bb5c-b2f3cd7e26dd None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:44.629808 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a54cf-0e86-4b0d-bb5c-b2f3cd7e26dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:44.648177 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-326a54cf-0e86-4b0d-bb5c-b2f3cd7e26dd None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:44.648391 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 767/1553] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:30:44.649371 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ac45f9-4a3d-4adf-b798-c02eeb28c62d None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:44.649501 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-326a54cf-0e86-4b0d-bb5c-b2f3cd7e26dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:44.662264 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ac45f9-4a3d-4adf-b798-c02eeb28c62d None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:44.715700 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 787/1554] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:30:44.717287 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa2e673-260d-49dc-838a-cb08a38ebc07 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:44.717583 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa2e673-260d-49dc-838a-cb08a38ebc07 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:44.717865 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa2e673-260d-49dc-838a-cb08a38ebc07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:44.723879 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54dcb7a-9d7a-4fd6-90d1-5c7892a2acaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:44.724561 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54dcb7a-9d7a-4fd6-90d1-5c7892a2acaf admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:44.724849 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54dcb7a-9d7a-4fd6-90d1-5c7892a2acaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:44.725133 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54dcb7a-9d7a-4fd6-90d1-5c7892a2acaf admin admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:44.725967 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54dcb7a-9d7a-4fd6-90d1-5c7892a2acaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:30:44.726151 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54dcb7a-9d7a-4fd6-90d1-5c7892a2acaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:44.727799 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54dcb7a-9d7a-4fd6-90d1-5c7892a2acaf admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:44.737570 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-dfa2e673-260d-49dc-838a-cb08a38ebc07 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:44.738810 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-dfa2e673-260d-49dc-838a-cb08a38ebc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:44.739546 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 768/1555] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:30:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:30:44.740784 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0519e5-da46-43ef-baa8-3abcfb1dab21 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:44.741081 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0519e5-da46-43ef-baa8-3abcfb1dab21 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:44.741365 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0519e5-da46-43ef-baa8-3abcfb1dab21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:44.755498 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 788/1556] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:30:44.756588 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa2e673-260d-49dc-838a-cb08a38ebc07 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:44.761431 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-2b0519e5-da46-43ef-baa8-3abcfb1dab21 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:44.762899 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-2b0519e5-da46-43ef-baa8-3abcfb1dab21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:44.767344 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa2e673-260d-49dc-838a-cb08a38ebc07 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:44.772131 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 769/1557] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:30:44.772891 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0519e5-da46-43ef-baa8-3abcfb1dab21 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:44.779522 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff8101-599d-44bd-822f-c6005e1316ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:44.780183 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff8101-599d-44bd-822f-c6005e1316ba admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:44.780472 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff8101-599d-44bd-822f-c6005e1316ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:44.780755 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff8101-599d-44bd-822f-c6005e1316ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:44.781461 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dff8101-599d-44bd-822f-c6005e1316ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:30:44.781770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dff8101-599d-44bd-822f-c6005e1316ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:44.782347 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0519e5-da46-43ef-baa8-3abcfb1dab21 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:44.783383 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dff8101-599d-44bd-822f-c6005e1316ba admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:44.787836 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd62f728-2a78-44bc-9596-d1e536530e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:44.788579 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62f728-2a78-44bc-9596-d1e536530e8c admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:44.788869 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62f728-2a78-44bc-9596-d1e536530e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:44.789145 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62f728-2a78-44bc-9596-d1e536530e8c admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:44.789832 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd62f728-2a78-44bc-9596-d1e536530e8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:44.791236 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: WARNING py.warnings [None req-fd62f728-2a78-44bc-9596-d1e536530e8c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 15:30:44.791236 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: warnings.warn(msg) Aug 05 15:30:44.791236 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: Aug 05 15:30:44.792207 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd62f728-2a78-44bc-9596-d1e536530e8c admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:44.795898 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 789/1558] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:30:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:30:44.796711 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff8101-599d-44bd-822f-c6005e1316ba admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:44.807096 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff8101-599d-44bd-822f-c6005e1316ba admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:44.811793 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a00f47-aa96-4de8-b96e-038e9260d4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:44.812423 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a00f47-aa96-4de8-b96e-038e9260d4dc admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:44.812687 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a00f47-aa96-4de8-b96e-038e9260d4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:44.812966 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a00f47-aa96-4de8-b96e-038e9260d4dc admin admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:44.813622 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a00f47-aa96-4de8-b96e-038e9260d4dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:44.815584 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a00f47-aa96-4de8-b96e-038e9260d4dc admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:44.877022 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:30:44.883008 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 770/1559] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:30:44 2022] POST /identity/v3/users => generated 290 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:30:44.883891 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd62f728-2a78-44bc-9596-d1e536530e8c admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:44.893603 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd62f728-2a78-44bc-9596-d1e536530e8c admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:44.894281 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 790/1560] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:30:44 2022] POST /identity/v3/users => generated 297 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:30:44.896443 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f922609-83ed-48d5-8b8b-c679ed410495 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:44.896734 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f922609-83ed-48d5-8b8b-c679ed410495 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:44.897014 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f922609-83ed-48d5-8b8b-c679ed410495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:44.899314 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02561a-c828-46aa-866d-3d1f07e19314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:44.899931 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02561a-c828-46aa-866d-3d1f07e19314 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:44.900262 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02561a-c828-46aa-866d-3d1f07e19314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:44.900545 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02561a-c828-46aa-866d-3d1f07e19314 admin admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:44.901243 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b02561a-c828-46aa-866d-3d1f07e19314 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:30:44.901557 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b02561a-c828-46aa-866d-3d1f07e19314 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:44.903155 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b02561a-c828-46aa-866d-3d1f07e19314 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:44.923348 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 771/1561] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:30:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:30:44.923534 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-0f922609-83ed-48d5-8b8b-c679ed410495 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:44.923534 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-0f922609-83ed-48d5-8b8b-c679ed410495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:44.923711 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02561a-c828-46aa-866d-3d1f07e19314 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:44.927003 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 791/1562] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:30:44.928251 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24193af5-24ba-4312-9de6-6d0efe1eb47a None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:44.928558 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24193af5-24ba-4312-9de6-6d0efe1eb47a None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:44.928831 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24193af5-24ba-4312-9de6-6d0efe1eb47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:44.929038 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02561a-c828-46aa-866d-3d1f07e19314 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:44.934734 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73fc7f5-562a-4ab7-be03-3b8a56baab9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:44.935367 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73fc7f5-562a-4ab7-be03-3b8a56baab9e admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:44.935667 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73fc7f5-562a-4ab7-be03-3b8a56baab9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:44.935946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73fc7f5-562a-4ab7-be03-3b8a56baab9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:44.936657 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73fc7f5-562a-4ab7-be03-3b8a56baab9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:44.938720 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73fc7f5-562a-4ab7-be03-3b8a56baab9e admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:44.948271 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-24193af5-24ba-4312-9de6-6d0efe1eb47a None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:44.949481 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-24193af5-24ba-4312-9de6-6d0efe1eb47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:44.958978 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 792/1563] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:30:44.960267 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c88de-1641-482c-ab05-787462e43284 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:44.960552 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c88de-1641-482c-ab05-787462e43284 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:44.960839 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c88de-1641-482c-ab05-787462e43284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:44.979721 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-b91c88de-1641-482c-ab05-787462e43284 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:44.980944 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-b91c88de-1641-482c-ab05-787462e43284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:44.988892 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 793/1564] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:30:44.989620 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91c88de-1641-482c-ab05-787462e43284 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:44.999279 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91c88de-1641-482c-ab05-787462e43284 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:45.004191 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7cc51c-31b3-435f-91bc-98f657354478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:45.005299 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7cc51c-31b3-435f-91bc-98f657354478 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:45.005626 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7cc51c-31b3-435f-91bc-98f657354478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:45.005908 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7cc51c-31b3-435f-91bc-98f657354478 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:45.006559 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7cc51c-31b3-435f-91bc-98f657354478 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:45.008509 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7cc51c-31b3-435f-91bc-98f657354478 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:45.015223 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 772/1565] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:30:44 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:30:45.024320 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73fc7f5-562a-4ab7-be03-3b8a56baab9e admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:45.083508 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73fc7f5-562a-4ab7-be03-3b8a56baab9e admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:45.088409 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c08d9-94f5-4a26-8e50-16c3b6d980ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:45.089019 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c08d9-94f5-4a26-8e50-16c3b6d980ea admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:45.089312 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c08d9-94f5-4a26-8e50-16c3b6d980ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:45.089642 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c08d9-94f5-4a26-8e50-16c3b6d980ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:45.090278 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082c08d9-94f5-4a26-8e50-16c3b6d980ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:45.091874 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082c08d9-94f5-4a26-8e50-16c3b6d980ea admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:45.094337 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 794/1566] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:30:44 2022] POST /identity/v3/projects => generated 365 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:30:45.095172 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7cc51c-31b3-435f-91bc-98f657354478 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:45.102922 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 773/1567] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:30:45.104632 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c08d9-94f5-4a26-8e50-16c3b6d980ea admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:45.111841 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:30:45.113496 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c08d9-94f5-4a26-8e50-16c3b6d980ea admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:45.162138 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7cc51c-31b3-435f-91bc-98f657354478 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:45.167549 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c953d4a-d3d1-47dd-a590-740802c49f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:45.168225 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c953d4a-d3d1-47dd-a590-740802c49f75 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:45.168519 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c953d4a-d3d1-47dd-a590-740802c49f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:45.168805 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c953d4a-d3d1-47dd-a590-740802c49f75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:45.169546 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c953d4a-d3d1-47dd-a590-740802c49f75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:45.171184 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c953d4a-d3d1-47dd-a590-740802c49f75 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:45.171691 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c9bcbf-b761-46da-8d17-ddcb8e0402ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:45.173274 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c9bcbf-b761-46da-8d17-ddcb8e0402ab admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:45.173716 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c9bcbf-b761-46da-8d17-ddcb8e0402ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:45.174154 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c9bcbf-b761-46da-8d17-ddcb8e0402ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:45.174883 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c9bcbf-b761-46da-8d17-ddcb8e0402ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:45.176816 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c9bcbf-b761-46da-8d17-ddcb8e0402ab admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:45.185288 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 795/1568] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:30:45.185288 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c953d4a-d3d1-47dd-a590-740802c49f75 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:45.193340 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c953d4a-d3d1-47dd-a590-740802c49f75 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:45.198018 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de063d3-c893-4aa5-89b5-aae2e0109035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:45.198625 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de063d3-c893-4aa5-89b5-aae2e0109035 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:45.198912 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de063d3-c893-4aa5-89b5-aae2e0109035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:45.199193 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de063d3-c893-4aa5-89b5-aae2e0109035 admin admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:45.199881 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de063d3-c893-4aa5-89b5-aae2e0109035 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:30:45.200241 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de063d3-c893-4aa5-89b5-aae2e0109035 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:45.202074 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de063d3-c893-4aa5-89b5-aae2e0109035 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:45.214523 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 796/1569] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:30:45.215353 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de063d3-c893-4aa5-89b5-aae2e0109035 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:45.225065 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de063d3-c893-4aa5-89b5-aae2e0109035 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:45.230817 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91e505-933f-468e-a4b4-0f2474d18ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:45.230817 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91e505-933f-468e-a4b4-0f2474d18ad6 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:45.230817 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91e505-933f-468e-a4b4-0f2474d18ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:45.230817 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91e505-933f-468e-a4b4-0f2474d18ad6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:45.231432 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91e505-933f-468e-a4b4-0f2474d18ad6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:30:45.231763 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91e505-933f-468e-a4b4-0f2474d18ad6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:45.233351 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91e505-933f-468e-a4b4-0f2474d18ad6 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:45.245587 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 797/1570] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:30:45.246348 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91e505-933f-468e-a4b4-0f2474d18ad6 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:45.256492 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91e505-933f-468e-a4b4-0f2474d18ad6 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:45.261193 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4cdc2-e675-4c6c-8653-291e6217bc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:45.261765 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4cdc2-e675-4c6c-8653-291e6217bc3d admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:45.262055 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4cdc2-e675-4c6c-8653-291e6217bc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:45.262336 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4cdc2-e675-4c6c-8653-291e6217bc3d admin admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:45.263010 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f4cdc2-e675-4c6c-8653-291e6217bc3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:30:45.263316 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f4cdc2-e675-4c6c-8653-291e6217bc3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:45.264908 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f4cdc2-e675-4c6c-8653-291e6217bc3d admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:45.276604 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 798/1571] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:30:45.277410 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4cdc2-e675-4c6c-8653-291e6217bc3d admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:45.283192 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 774/1572] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:30:45 2022] POST /identity/v3/projects => generated 387 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:30:45.283844 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c9bcbf-b761-46da-8d17-ddcb8e0402ab admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:45.289463 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4cdc2-e675-4c6c-8653-291e6217bc3d admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:45.331175 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:30:45.345573 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c9bcbf-b761-46da-8d17-ddcb8e0402ab admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:45.346543 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5843d2d-d484-414c-ad67-16b250f82a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:45.347183 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5843d2d-d484-414c-ad67-16b250f82a3b admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:45.347455 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5843d2d-d484-414c-ad67-16b250f82a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:45.347734 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5843d2d-d484-414c-ad67-16b250f82a3b admin admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:45.348435 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5843d2d-d484-414c-ad67-16b250f82a3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:45.350278 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e079932-7bc8-4d71-b446-0f7aa1f38b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:45.350402 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5843d2d-d484-414c-ad67-16b250f82a3b admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:45.350902 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e079932-7bc8-4d71-b446-0f7aa1f38b13 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:45.351186 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e079932-7bc8-4d71-b446-0f7aa1f38b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:45.351467 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e079932-7bc8-4d71-b446-0f7aa1f38b13 admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:45.352161 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e079932-7bc8-4d71-b446-0f7aa1f38b13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:45.354454 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e079932-7bc8-4d71-b446-0f7aa1f38b13 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:45.447094 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 799/1573] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:30:45 2022] POST /identity/v3/users => generated 289 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:30:45.447821 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5843d2d-d484-414c-ad67-16b250f82a3b admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:45.457092 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5843d2d-d484-414c-ad67-16b250f82a3b admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:45.462919 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 775/1574] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:30:45 2022] POST /identity/v3/users => generated 283 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:30:45.462919 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e079932-7bc8-4d71-b446-0f7aa1f38b13 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:45.465668 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c44bd-aa74-4b8f-806f-b36dc0d044ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:45.466270 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c44bd-aa74-4b8f-806f-b36dc0d044ed admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:45.466563 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c44bd-aa74-4b8f-806f-b36dc0d044ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:45.466836 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c44bd-aa74-4b8f-806f-b36dc0d044ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:45.467521 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61c44bd-aa74-4b8f-806f-b36dc0d044ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:45.469158 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61c44bd-aa74-4b8f-806f-b36dc0d044ed admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:45.470336 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e079932-7bc8-4d71-b446-0f7aa1f38b13 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:45.475656 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01507df7-151b-47ee-81c4-bbf3c6c9d47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:45.476295 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01507df7-151b-47ee-81c4-bbf3c6c9d47d admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:45.478441 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01507df7-151b-47ee-81c4-bbf3c6c9d47d admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:45.478441 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01507df7-151b-47ee-81c4-bbf3c6c9d47d admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:45.478441 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01507df7-151b-47ee-81c4-bbf3c6c9d47d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:45.479142 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01507df7-151b-47ee-81c4-bbf3c6c9d47d admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:45.479728 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 800/1575] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:30:45.480583 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c44bd-aa74-4b8f-806f-b36dc0d044ed admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:45.489783 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c44bd-aa74-4b8f-806f-b36dc0d044ed admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:45.492305 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 776/1576] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:30:45.493283 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01507df7-151b-47ee-81c4-bbf3c6c9d47d admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:45.494674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73809d7-8fc0-4c2e-8f5f-43e824204062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:45.495248 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73809d7-8fc0-4c2e-8f5f-43e824204062 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:45.495555 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73809d7-8fc0-4c2e-8f5f-43e824204062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:45.495806 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73809d7-8fc0-4c2e-8f5f-43e824204062 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:45.496485 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73809d7-8fc0-4c2e-8f5f-43e824204062 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:45.498409 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73809d7-8fc0-4c2e-8f5f-43e824204062 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:45.503365 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01507df7-151b-47ee-81c4-bbf3c6c9d47d admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:45.508057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d70182-a6f3-4a4d-b258-7d0eb0c956d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:45.508671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d70182-a6f3-4a4d-b258-7d0eb0c956d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:45.508958 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d70182-a6f3-4a4d-b258-7d0eb0c956d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:45.509234 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d70182-a6f3-4a4d-b258-7d0eb0c956d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:45.509924 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d70182-a6f3-4a4d-b258-7d0eb0c956d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:45.511802 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d70182-a6f3-4a4d-b258-7d0eb0c956d7 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:45.611693 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 801/1577] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:30:45 2022] POST /identity/v3/projects => generated 373 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:30:45.611693 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73809d7-8fc0-4c2e-8f5f-43e824204062 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:45.621042 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 777/1578] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:30:45 2022] POST /identity/v3/projects => generated 403 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:30:45.623116 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d70182-a6f3-4a4d-b258-7d0eb0c956d7 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:45.640830 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:30:45.673396 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73809d7-8fc0-4c2e-8f5f-43e824204062 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:45.685520 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d70182-a6f3-4a4d-b258-7d0eb0c956d7 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:45.690303 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0109e26-6684-433a-9dda-a508513e3223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:45.690917 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0109e26-6684-433a-9dda-a508513e3223 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:45.691200 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0109e26-6684-433a-9dda-a508513e3223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:45.691474 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0109e26-6684-433a-9dda-a508513e3223 admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:45.692760 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0109e26-6684-433a-9dda-a508513e3223 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:45.694674 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0109e26-6684-433a-9dda-a508513e3223 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:45.730001 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f2ccc-c6ad-43c7-9b3a-b3c6a93bb62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:45.730551 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f2ccc-c6ad-43c7-9b3a-b3c6a93bb62a admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:45.730838 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f2ccc-c6ad-43c7-9b3a-b3c6a93bb62a admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:45.731120 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f2ccc-c6ad-43c7-9b3a-b3c6a93bb62a admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:45.731787 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9f2ccc-c6ad-43c7-9b3a-b3c6a93bb62a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:45.733738 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9f2ccc-c6ad-43c7-9b3a-b3c6a93bb62a admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:45.801484 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 778/1579] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:30:45 2022] POST /identity/v3/users => generated 294 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:30:45.802353 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0109e26-6684-433a-9dda-a508513e3223 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:45.811693 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0109e26-6684-433a-9dda-a508513e3223 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:45.816353 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a626c4-c412-4d8e-85bf-24881a03debb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:45.817489 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a626c4-c412-4d8e-85bf-24881a03debb admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:45.817793 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a626c4-c412-4d8e-85bf-24881a03debb admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:45.818062 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a626c4-c412-4d8e-85bf-24881a03debb admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:45.818742 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a626c4-c412-4d8e-85bf-24881a03debb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:45.820366 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a626c4-c412-4d8e-85bf-24881a03debb admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:45.830546 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 779/1580] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:30:45.831349 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a626c4-c412-4d8e-85bf-24881a03debb admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:45.835171 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 802/1581] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:30:45 2022] POST /identity/v3/projects => generated 371 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:30:45.836007 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f2ccc-c6ad-43c7-9b3a-b3c6a93bb62a admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:45.843111 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a626c4-c412-4d8e-85bf-24881a03debb admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:45.898982 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f2ccc-c6ad-43c7-9b3a-b3c6a93bb62a admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:45.899410 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1cb931-8bfb-43b5-bd64-a18208640f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:45.900036 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1cb931-8bfb-43b5-bd64-a18208640f0e admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:45.900353 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1cb931-8bfb-43b5-bd64-a18208640f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:45.900634 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1cb931-8bfb-43b5-bd64-a18208640f0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:45.901340 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1cb931-8bfb-43b5-bd64-a18208640f0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:45.902904 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1cb931-8bfb-43b5-bd64-a18208640f0e admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:45.903662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ae1ce-4291-4769-a45a-f50622b9a409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:45.904436 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ae1ce-4291-4769-a45a-f50622b9a409 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:45.904718 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ae1ce-4291-4769-a45a-f50622b9a409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:45.905119 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ae1ce-4291-4769-a45a-f50622b9a409 admin admin] PATH_INFO: `/v3/projects/020b2f0e82d14211b286f1ac1e3823a9/users/b0882cb9bf4744feae298fb8f10cebeb/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:45.914706 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 780/1582] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:30:45.915881 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1cb931-8bfb-43b5-bd64-a18208640f0e admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:45.920222 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136ae1ce-4291-4769-a45a-f50622b9a409 admin admin] RBAC: Authorizing `identity:create_grant(project_id=020b2f0e82d14211b286f1ac1e3823a9, user_id=b0882cb9bf4744feae298fb8f10cebeb, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:45.921446 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: WARNING py.warnings [None req-136ae1ce-4291-4769-a45a-f50622b9a409 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 15:30:45.921446 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: warnings.warn(msg) Aug 05 15:30:45.921446 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: Aug 05 15:30:45.923404 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136ae1ce-4291-4769-a45a-f50622b9a409 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:45.926008 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1cb931-8bfb-43b5-bd64-a18208640f0e admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:45.930750 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de75518-66af-4791-80dd-af0f5e9c6c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:45.938558 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de75518-66af-4791-80dd-af0f5e9c6c60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:45.938558 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de75518-66af-4791-80dd-af0f5e9c6c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:45.938558 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de75518-66af-4791-80dd-af0f5e9c6c60 admin admin] PATH_INFO: `/v3/projects/1cb48dbd26c747d290bf3e666e65027a/users/cb055110c77543c1861dfd8c155d7582/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:45.946864 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de75518-66af-4791-80dd-af0f5e9c6c60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cb48dbd26c747d290bf3e666e65027a, user_id=cb055110c77543c1861dfd8c155d7582, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:45.949537 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de75518-66af-4791-80dd-af0f5e9c6c60 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:45.951726 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:30:46.034527 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 803/1583] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:30:45 2022] PUT /identity/v3/projects/020b2f0e82d14211b286f1ac1e3823a9/users/b0882cb9bf4744feae298fb8f10cebeb/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:30:46.035377 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ae1ce-4291-4769-a45a-f50622b9a409 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:46.082322 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 781/1584] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:30:45 2022] PUT /identity/v3/projects/1cb48dbd26c747d290bf3e666e65027a/users/cb055110c77543c1861dfd8c155d7582/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:30:46.083188 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de75518-66af-4791-80dd-af0f5e9c6c60 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:46.095505 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:30:46.113227 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ae1ce-4291-4769-a45a-f50622b9a409 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:46.146502 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de75518-66af-4791-80dd-af0f5e9c6c60 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:46.151082 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4defb97a-140c-4930-91d1-fb8e4704c327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:46.151695 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4defb97a-140c-4930-91d1-fb8e4704c327 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:46.151976 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4defb97a-140c-4930-91d1-fb8e4704c327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:46.152280 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4defb97a-140c-4930-91d1-fb8e4704c327 admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:46.152936 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4defb97a-140c-4930-91d1-fb8e4704c327 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:46.154850 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4defb97a-140c-4930-91d1-fb8e4704c327 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:46.173163 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3585a3-6e5d-498f-905c-938060c43bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:46.173618 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3585a3-6e5d-498f-905c-938060c43bf4 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:46.173903 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3585a3-6e5d-498f-905c-938060c43bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:46.174496 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3585a3-6e5d-498f-905c-938060c43bf4 admin admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:46.174882 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3585a3-6e5d-498f-905c-938060c43bf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:46.177545 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3585a3-6e5d-498f-905c-938060c43bf4 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:46.274840 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 782/1585] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:30:46 2022] POST /identity/v3/users => generated 302 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:30:46.275714 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4defb97a-140c-4930-91d1-fb8e4704c327 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:46.285268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4defb97a-140c-4930-91d1-fb8e4704c327 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:46.289843 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b92bf2-7bc6-41f8-8cbc-be3900e75750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:46.290419 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b92bf2-7bc6-41f8-8cbc-be3900e75750 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:46.290705 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b92bf2-7bc6-41f8-8cbc-be3900e75750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:46.290986 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b92bf2-7bc6-41f8-8cbc-be3900e75750 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:46.291670 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b92bf2-7bc6-41f8-8cbc-be3900e75750 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:46.293318 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b92bf2-7bc6-41f8-8cbc-be3900e75750 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:46.304151 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 804/1586] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:30:46 2022] POST /identity/v3/users => generated 287 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:30:46.307469 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 783/1587] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:30:46.307469 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b92bf2-7bc6-41f8-8cbc-be3900e75750 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:46.307684 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3585a3-6e5d-498f-905c-938060c43bf4 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:46.315389 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3585a3-6e5d-498f-905c-938060c43bf4 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:46.317999 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b92bf2-7bc6-41f8-8cbc-be3900e75750 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:46.320768 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec3af34-bc65-46d2-996b-6108f9fbe206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:46.321370 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec3af34-bc65-46d2-996b-6108f9fbe206 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:46.321653 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec3af34-bc65-46d2-996b-6108f9fbe206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:46.321960 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec3af34-bc65-46d2-996b-6108f9fbe206 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:46.322647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec3af34-bc65-46d2-996b-6108f9fbe206 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:46.322786 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6f46be-ffe4-4adb-9974-da65dd0cc19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:46.323429 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6f46be-ffe4-4adb-9974-da65dd0cc19d admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:46.323716 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6f46be-ffe4-4adb-9974-da65dd0cc19d admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:46.324027 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6f46be-ffe4-4adb-9974-da65dd0cc19d admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:46.324277 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec3af34-bc65-46d2-996b-6108f9fbe206 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:46.324786 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6f46be-ffe4-4adb-9974-da65dd0cc19d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:46.326699 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6f46be-ffe4-4adb-9974-da65dd0cc19d admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:46.334220 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 805/1588] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:30:46.335011 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec3af34-bc65-46d2-996b-6108f9fbe206 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:46.344505 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec3af34-bc65-46d2-996b-6108f9fbe206 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:46.349621 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549d3d5b-e630-4de6-b127-fe3ee373c29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:46.350219 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d3d5b-e630-4de6-b127-fe3ee373c29f admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:46.350501 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d3d5b-e630-4de6-b127-fe3ee373c29f admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:46.350776 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d3d5b-e630-4de6-b127-fe3ee373c29f admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:46.351456 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549d3d5b-e630-4de6-b127-fe3ee373c29f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:46.353057 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549d3d5b-e630-4de6-b127-fe3ee373c29f admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:46.364515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 806/1589] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:30:46.365733 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9031730-9d3c-4a4b-b978-eacce06213b0 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:46.366029 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9031730-9d3c-4a4b-b978-eacce06213b0 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:46.366332 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9031730-9d3c-4a4b-b978-eacce06213b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:46.384509 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-d9031730-9d3c-4a4b-b978-eacce06213b0 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:46.385718 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-d9031730-9d3c-4a4b-b978-eacce06213b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:46.393250 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 807/1590] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:30:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:30:46.394525 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4ffe1-1f41-4268-a000-bcf18a4e6ed4 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:46.394816 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4ffe1-1f41-4268-a000-bcf18a4e6ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:46.395090 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4ffe1-1f41-4268-a000-bcf18a4e6ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:46.414018 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 784/1591] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:30:46 2022] POST /identity/v3/users => generated 286 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:30:46.414018 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6f46be-ffe4-4adb-9974-da65dd0cc19d admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:46.423791 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-30f4ffe1-1f41-4268-a000-bcf18a4e6ed4 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:46.423791 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-30f4ffe1-1f41-4268-a000-bcf18a4e6ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:46.425119 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6f46be-ffe4-4adb-9974-da65dd0cc19d admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:46.428774 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 808/1592] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:30:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:30:46.429550 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f4ffe1-1f41-4268-a000-bcf18a4e6ed4 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:46.439724 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f4ffe1-1f41-4268-a000-bcf18a4e6ed4 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:46.441430 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3d3c7-eb83-4d3c-b2fd-34c4670e2029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:46.442069 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3d3c7-eb83-4d3c-b2fd-34c4670e2029 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:46.442357 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3d3c7-eb83-4d3c-b2fd-34c4670e2029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:46.442638 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3d3c7-eb83-4d3c-b2fd-34c4670e2029 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:46.443339 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f3d3c7-eb83-4d3c-b2fd-34c4670e2029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:46.444424 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5257382-37e4-4bb7-92c2-983277baa8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:46.449743 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f3d3c7-eb83-4d3c-b2fd-34c4670e2029 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:46.450625 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5257382-37e4-4bb7-92c2-983277baa8f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:46.450625 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5257382-37e4-4bb7-92c2-983277baa8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:46.450625 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5257382-37e4-4bb7-92c2-983277baa8f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:46.450625 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5257382-37e4-4bb7-92c2-983277baa8f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:46.450625 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5257382-37e4-4bb7-92c2-983277baa8f0 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:46.457986 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 785/1593] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:30:46.461318 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 809/1594] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:30:46.461431 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3d3c7-eb83-4d3c-b2fd-34c4670e2029 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:46.461541 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5257382-37e4-4bb7-92c2-983277baa8f0 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:46.470688 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3d3c7-eb83-4d3c-b2fd-34c4670e2029 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:46.471368 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5257382-37e4-4bb7-92c2-983277baa8f0 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:46.475760 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb268cd5-802f-4ecc-9a89-d1561fc42dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:46.476387 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb268cd5-802f-4ecc-9a89-d1561fc42dd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:46.476503 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110d3bc-49cd-4457-9884-c43dc78e931b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:46.476668 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb268cd5-802f-4ecc-9a89-d1561fc42dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:46.476946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb268cd5-802f-4ecc-9a89-d1561fc42dd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:46.477052 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110d3bc-49cd-4457-9884-c43dc78e931b admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:46.477325 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110d3bc-49cd-4457-9884-c43dc78e931b admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:46.477604 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110d3bc-49cd-4457-9884-c43dc78e931b admin admin] PATH_INFO: `/v3/projects/f9776de9e6d341fdbf7ac4fa70b994aa/users/04e80222936d4427909414a78e1dc489/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:46.477710 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb268cd5-802f-4ecc-9a89-d1561fc42dd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:46.479180 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb268cd5-802f-4ecc-9a89-d1561fc42dd1 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:46.489742 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 786/1595] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:30:46.490841 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb268cd5-802f-4ecc-9a89-d1561fc42dd1 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:46.496013 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9110d3bc-49cd-4457-9884-c43dc78e931b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9776de9e6d341fdbf7ac4fa70b994aa, user_id=04e80222936d4427909414a78e1dc489, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:46.498688 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9110d3bc-49cd-4457-9884-c43dc78e931b admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:46.500574 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb268cd5-802f-4ecc-9a89-d1561fc42dd1 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:46.505140 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5997207d-4951-4be1-99b3-c3cbde75ab77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:46.505783 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5997207d-4951-4be1-99b3-c3cbde75ab77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:46.506068 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5997207d-4951-4be1-99b3-c3cbde75ab77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:46.506347 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5997207d-4951-4be1-99b3-c3cbde75ab77 admin admin] PATH_INFO: `/v3/projects/383dd2344dc4403aa7f4f0953435ddc3/users/3e26066d56b74a3ca5fed6b180f5c689/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:46.520698 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5997207d-4951-4be1-99b3-c3cbde75ab77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=383dd2344dc4403aa7f4f0953435ddc3, user_id=3e26066d56b74a3ca5fed6b180f5c689, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:46.523353 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5997207d-4951-4be1-99b3-c3cbde75ab77 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:46.595079 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 810/1596] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:30:46 2022] PUT /identity/v3/projects/f9776de9e6d341fdbf7ac4fa70b994aa/users/04e80222936d4427909414a78e1dc489/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:30:46.595975 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110d3bc-49cd-4457-9884-c43dc78e931b admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:46.606992 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 787/1597] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:30:46 2022] PUT /identity/v3/projects/383dd2344dc4403aa7f4f0953435ddc3/users/3e26066d56b74a3ca5fed6b180f5c689/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:30:46.607818 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5997207d-4951-4be1-99b3-c3cbde75ab77 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:46.652748 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110d3bc-49cd-4457-9884-c43dc78e931b admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:46.672396 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5997207d-4951-4be1-99b3-c3cbde75ab77 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:46.673240 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:30:46.678154 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e4e90-410b-4083-bc67-050203185618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:46.678821 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e4e90-410b-4083-bc67-050203185618 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:46.679080 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e4e90-410b-4083-bc67-050203185618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:46.679356 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e4e90-410b-4083-bc67-050203185618 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:46.680048 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7e4e90-410b-4083-bc67-050203185618 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:46.684466 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7e4e90-410b-4083-bc67-050203185618 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:46.691567 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 788/1598] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:30:46.692483 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e4e90-410b-4083-bc67-050203185618 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:46.697969 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:30:46.703619 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c4d1e-71be-4a72-9546-7e4d86665889 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:46.704444 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c4d1e-71be-4a72-9546-7e4d86665889 service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:46.705114 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c4d1e-71be-4a72-9546-7e4d86665889 service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:46.705460 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c4d1e-71be-4a72-9546-7e4d86665889 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:46.715746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8c4d1e-71be-4a72-9546-7e4d86665889 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:46.718509 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8c4d1e-71be-4a72-9546-7e4d86665889 service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:46.733153 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 811/1599] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:30:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:30:46.734514 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c4d1e-71be-4a72-9546-7e4d86665889 service neutron] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:46.744531 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c4d1e-71be-4a72-9546-7e4d86665889 service neutron] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:46.749498 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d492d21-25b3-4282-8ac9-58e37c9bda6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:46.750121 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d492d21-25b3-4282-8ac9-58e37c9bda6c admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:46.750412 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d492d21-25b3-4282-8ac9-58e37c9bda6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:46.750695 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d492d21-25b3-4282-8ac9-58e37c9bda6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:46.751529 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d492d21-25b3-4282-8ac9-58e37c9bda6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:46.751893 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e4e90-410b-4083-bc67-050203185618 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:46.753360 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d492d21-25b3-4282-8ac9-58e37c9bda6c admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:46.755613 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf80bf6-8cfe-48ce-b634-a29e60fa550d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:46.756272 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf80bf6-8cfe-48ce-b634-a29e60fa550d service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:46.756727 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf80bf6-8cfe-48ce-b634-a29e60fa550d service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:46.757135 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf80bf6-8cfe-48ce-b634-a29e60fa550d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:46.766504 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf80bf6-8cfe-48ce-b634-a29e60fa550d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:46.768927 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 812/1600] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:30:46.768927 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d492d21-25b3-4282-8ac9-58e37c9bda6c admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:46.769146 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf80bf6-8cfe-48ce-b634-a29e60fa550d service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:46.778648 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 789/1601] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:30:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:30:46.779512 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf80bf6-8cfe-48ce-b634-a29e60fa550d service neutron] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:46.780175 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d492d21-25b3-4282-8ac9-58e37c9bda6c admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:46.785021 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f613b41f-d816-4c24-971b-30849ea5e926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:46.786337 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613b41f-d816-4c24-971b-30849ea5e926 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:46.786620 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613b41f-d816-4c24-971b-30849ea5e926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:46.786903 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613b41f-d816-4c24-971b-30849ea5e926 admin admin] PATH_INFO: `/v3/projects/648f7ac8fb934441b6782d00d0b5ef10/users/d36b38a6e2d34c41b827ff6db9742332/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:46.789428 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf80bf6-8cfe-48ce-b634-a29e60fa550d service neutron] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:46.795073 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fada95-2d2e-46c5-b55a-65446b14d115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:46.803459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fada95-2d2e-46c5-b55a-65446b14d115 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:46.803459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fada95-2d2e-46c5-b55a-65446b14d115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:46.803459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fada95-2d2e-46c5-b55a-65446b14d115 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:46.803459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fada95-2d2e-46c5-b55a-65446b14d115 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:46.803459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fada95-2d2e-46c5-b55a-65446b14d115 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:46.806440 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f613b41f-d816-4c24-971b-30849ea5e926 admin admin] RBAC: Authorizing `identity:create_grant(project_id=648f7ac8fb934441b6782d00d0b5ef10, user_id=d36b38a6e2d34c41b827ff6db9742332, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:46.808761 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 790/1602] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:30:46.809717 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f613b41f-d816-4c24-971b-30849ea5e926 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:46.813308 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e6644-dc15-4cfe-8ece-0d5693b82bea None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:46.813579 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e6644-dc15-4cfe-8ece-0d5693b82bea None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:46.813858 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e6644-dc15-4cfe-8ece-0d5693b82bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:46.832178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-fb2e6644-dc15-4cfe-8ece-0d5693b82bea None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:46.833935 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-fb2e6644-dc15-4cfe-8ece-0d5693b82bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:46.841964 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 791/1603] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:30:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:30:46.843184 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a618a7-4e49-4735-b8ec-99af3dd60577 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:46.843474 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a618a7-4e49-4735-b8ec-99af3dd60577 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:46.843756 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a618a7-4e49-4735-b8ec-99af3dd60577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:46.873274 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-b5a618a7-4e49-4735-b8ec-99af3dd60577 None None] MFA Rules not processed for user `3e26066d56b74a3ca5fed6b180f5c689`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:46.874515 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-b5a618a7-4e49-4735-b8ec-99af3dd60577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:46.940111 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 813/1604] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:30:46 2022] PUT /identity/v3/projects/648f7ac8fb934441b6782d00d0b5ef10/users/d36b38a6e2d34c41b827ff6db9742332/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:30:46.948361 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f613b41f-d816-4c24-971b-30849ea5e926 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:46.955423 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 792/1605] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:30:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:30:46.955423 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a618a7-4e49-4735-b8ec-99af3dd60577 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:46.967660 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a618a7-4e49-4735-b8ec-99af3dd60577 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:47.023272 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f613b41f-d816-4c24-971b-30849ea5e926 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:47.028982 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6db5ba-8148-4a02-9be4-92862a65c1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:47.033036 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6db5ba-8148-4a02-9be4-92862a65c1e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:47.033036 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6db5ba-8148-4a02-9be4-92862a65c1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:47.033036 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6db5ba-8148-4a02-9be4-92862a65c1e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:47.033036 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6db5ba-8148-4a02-9be4-92862a65c1e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:47.033036 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6db5ba-8148-4a02-9be4-92862a65c1e3 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:47.037302 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1111c5-a95c-4b9b-90d9-917dee10ae5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:47.038051 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1111c5-a95c-4b9b-90d9-917dee10ae5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:47.039100 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1111c5-a95c-4b9b-90d9-917dee10ae5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:47.039100 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1111c5-a95c-4b9b-90d9-917dee10ae5a admin admin] PATH_INFO: `/v3/projects/75f1e9851c4945a599dd0843f17c2530/users/ec7fc07d65854cb18789d6cfe8e7d710/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:47.049855 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 814/1606] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:30:47.051693 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6db5ba-8148-4a02-9be4-92862a65c1e3 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:47.062612 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1111c5-a95c-4b9b-90d9-917dee10ae5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=75f1e9851c4945a599dd0843f17c2530, user_id=ec7fc07d65854cb18789d6cfe8e7d710, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:47.064216 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6db5ba-8148-4a02-9be4-92862a65c1e3 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:47.065484 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1111c5-a95c-4b9b-90d9-917dee10ae5a admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:47.069865 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b665f0-d80c-4bfa-8741-af2e4840e83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:47.070578 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b665f0-d80c-4bfa-8741-af2e4840e83a admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:47.070865 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b665f0-d80c-4bfa-8741-af2e4840e83a admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:47.071283 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b665f0-d80c-4bfa-8741-af2e4840e83a admin admin] PATH_INFO: `/v3/projects/521158d79b45465298460c240f399055/users/be954cca051f450f845c206c7655d841/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:47.090355 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b665f0-d80c-4bfa-8741-af2e4840e83a admin admin] RBAC: Authorizing `identity:create_grant(project_id=521158d79b45465298460c240f399055, user_id=be954cca051f450f845c206c7655d841, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:47.093136 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b665f0-d80c-4bfa-8741-af2e4840e83a admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:47.136700 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 793/1607] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:30:46 2022] PUT /identity/v3/projects/75f1e9851c4945a599dd0843f17c2530/users/ec7fc07d65854cb18789d6cfe8e7d710/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:30:47.136700 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1111c5-a95c-4b9b-90d9-917dee10ae5a admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:47.160025 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:30:47.162362 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 815/1608] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:30:47 2022] PUT /identity/v3/projects/521158d79b45465298460c240f399055/users/be954cca051f450f845c206c7655d841/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:30:47.164437 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a4a39-6faa-43f9-98d4-c96ab9dd8a5c None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:47.165482 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a4a39-6faa-43f9-98d4-c96ab9dd8a5c None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:47.165971 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a4a39-6faa-43f9-98d4-c96ab9dd8a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:47.192624 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-e25a4a39-6faa-43f9-98d4-c96ab9dd8a5c None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:47.192624 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-e25a4a39-6faa-43f9-98d4-c96ab9dd8a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:47.235364 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1111c5-a95c-4b9b-90d9-917dee10ae5a admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:47.264292 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 816/1609] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:30:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:30:47.265238 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25a4a39-6faa-43f9-98d4-c96ab9dd8a5c None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:47.278310 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:30:47.293405 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b799f0-1292-456d-9b11-e5ab05ae766f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:47.294080 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b799f0-1292-456d-9b11-e5ab05ae766f service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:47.294316 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b799f0-1292-456d-9b11-e5ab05ae766f service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:47.294627 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b799f0-1292-456d-9b11-e5ab05ae766f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:47.304580 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b799f0-1292-456d-9b11-e5ab05ae766f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:47.307248 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b799f0-1292-456d-9b11-e5ab05ae766f service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:47.318165 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 794/1610] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:30:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:30:47.318868 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b799f0-1292-456d-9b11-e5ab05ae766f service neutron] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:47.329234 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b799f0-1292-456d-9b11-e5ab05ae766f service neutron] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:47.340489 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826f3961-3bd4-445a-9276-f3a213325838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:47.341244 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f3961-3bd4-445a-9276-f3a213325838 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:47.341590 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f3961-3bd4-445a-9276-f3a213325838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:47.341881 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f3961-3bd4-445a-9276-f3a213325838 admin admin] PATH_INFO: `/v3/projects/2ff2e00771274e4ea0ca0198dabb11a1/users/9c34950ab0aa4affb28d38ba50fa93c8/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:47.368303 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826f3961-3bd4-445a-9276-f3a213325838 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ff2e00771274e4ea0ca0198dabb11a1, user_id=9c34950ab0aa4affb28d38ba50fa93c8, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:47.368734 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25a4a39-6faa-43f9-98d4-c96ab9dd8a5c None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:47.372944 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826f3961-3bd4-445a-9276-f3a213325838 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:47.373520 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a09308a-eb2a-4f63-950b-fbc7a76a7631 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:47.374595 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a09308a-eb2a-4f63-950b-fbc7a76a7631 service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:47.375014 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a09308a-eb2a-4f63-950b-fbc7a76a7631 service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:47.375391 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a09308a-eb2a-4f63-950b-fbc7a76a7631 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:47.443861 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a09308a-eb2a-4f63-950b-fbc7a76a7631 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:47.445573 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a09308a-eb2a-4f63-950b-fbc7a76a7631 service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:47.454820 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 817/1611] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:30:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:30:47.455951 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d668d46b-b195-4a27-b56d-9b63abe0c1de None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:47.456374 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d668d46b-b195-4a27-b56d-9b63abe0c1de None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:47.456736 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d668d46b-b195-4a27-b56d-9b63abe0c1de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:47.471201 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 795/1612] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:30:47 2022] PUT /identity/v3/projects/2ff2e00771274e4ea0ca0198dabb11a1/users/9c34950ab0aa4affb28d38ba50fa93c8/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:30:47.472501 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ecf8e-5eb1-438f-b687-d23d6df84369 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:47.472791 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ecf8e-5eb1-438f-b687-d23d6df84369 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:47.473105 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ecf8e-5eb1-438f-b687-d23d6df84369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:47.492520 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-d668d46b-b195-4a27-b56d-9b63abe0c1de None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:47.494886 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-d668d46b-b195-4a27-b56d-9b63abe0c1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:47.533995 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-9b3ecf8e-5eb1-438f-b687-d23d6df84369 None None] MFA Rules not processed for user `be954cca051f450f845c206c7655d841`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:47.537020 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-9b3ecf8e-5eb1-438f-b687-d23d6df84369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:47.636515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 818/1613] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:30:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:30:47.637467 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d668d46b-b195-4a27-b56d-9b63abe0c1de None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:47.683723 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 796/1614] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:30:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:30:47.685155 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807da40-cc46-42b4-a30a-8c1628b5cc52 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:47.685452 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807da40-cc46-42b4-a30a-8c1628b5cc52 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:47.685739 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807da40-cc46-42b4-a30a-8c1628b5cc52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:47.740254 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d668d46b-b195-4a27-b56d-9b63abe0c1de None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:47.744048 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-6807da40-cc46-42b4-a30a-8c1628b5cc52 None None] MFA Rules not processed for user `9c34950ab0aa4affb28d38ba50fa93c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:47.745379 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-6807da40-cc46-42b4-a30a-8c1628b5cc52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:47.751013 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17628988-434f-432a-922d-bb97dde8f127 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:47.751930 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17628988-434f-432a-922d-bb97dde8f127 service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:47.752871 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17628988-434f-432a-922d-bb97dde8f127 service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:47.755446 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17628988-434f-432a-922d-bb97dde8f127 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:47.761112 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:30:47.782029 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17628988-434f-432a-922d-bb97dde8f127 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:47.783802 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17628988-434f-432a-922d-bb97dde8f127 service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:47.802072 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 819/1615] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:30:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:30:47.805635 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17628988-434f-432a-922d-bb97dde8f127 service neutron] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:47.809840 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:30:47.817988 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17628988-434f-432a-922d-bb97dde8f127 service neutron] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:47.822040 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3232c0-7b4d-435c-bc9e-52437c49f91f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:47.822667 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3232c0-7b4d-435c-bc9e-52437c49f91f service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:47.823298 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3232c0-7b4d-435c-bc9e-52437c49f91f service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:47.823596 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3232c0-7b4d-435c-bc9e-52437c49f91f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:47.834992 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3232c0-7b4d-435c-bc9e-52437c49f91f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:47.836685 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3232c0-7b4d-435c-bc9e-52437c49f91f service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:47.847368 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 797/1616] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:30:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:30:47.850409 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77fc50-90d3-459f-a347-19ac1117feb4 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:47.850409 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77fc50-90d3-459f-a347-19ac1117feb4 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:47.850799 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77fc50-90d3-459f-a347-19ac1117feb4 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:47.853044 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 820/1617] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:30:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:30:47.854128 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3232c0-7b4d-435c-bc9e-52437c49f91f service neutron] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:47.854264 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 798/1618] 10.222.0.40 () {54 vars in 962 bytes} [Fri Aug 5 15:30:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:30:47.861327 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9494a7c1-9a59-454d-8cf0-37c30dca72d4 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:47.864369 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9494a7c1-9a59-454d-8cf0-37c30dca72d4 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:47.864369 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9494a7c1-9a59-454d-8cf0-37c30dca72d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:47.866100 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3232c0-7b4d-435c-bc9e-52437c49f91f service neutron] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:47.871568 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6536e39-9964-432e-9e86-e177d3cf27e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:47.872349 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6536e39-9964-432e-9e86-e177d3cf27e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:47.872792 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6536e39-9964-432e-9e86-e177d3cf27e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:47.872918 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6536e39-9964-432e-9e86-e177d3cf27e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:47.873762 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6536e39-9964-432e-9e86-e177d3cf27e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:47.875873 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6536e39-9964-432e-9e86-e177d3cf27e3 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:47.927164 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-9494a7c1-9a59-454d-8cf0-37c30dca72d4 None None] MFA Rules not processed for user `bc0497dab1f44bb7b8196d39bf01d770`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:47.928512 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-9494a7c1-9a59-454d-8cf0-37c30dca72d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:47.976851 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 821/1619] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:30:47 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:30:47.982019 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6536e39-9964-432e-9e86-e177d3cf27e3 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:48.041012 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 799/1620] 10.222.0.40 () {58 vars in 1059 bytes} [Fri Aug 5 15:30:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:30:48.049011 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9494a7c1-9a59-454d-8cf0-37c30dca72d4 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:48.067147 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6536e39-9964-432e-9e86-e177d3cf27e3 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:48.070584 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9494a7c1-9a59-454d-8cf0-37c30dca72d4 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:48.077723 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921a8aa7-21a7-48dd-bc18-a7a8210a246a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:48.077723 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a8aa7-21a7-48dd-bc18-a7a8210a246a admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:48.077723 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a8aa7-21a7-48dd-bc18-a7a8210a246a admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:48.077723 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a8aa7-21a7-48dd-bc18-a7a8210a246a admin admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:48.078100 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921a8aa7-21a7-48dd-bc18-a7a8210a246a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:48.078655 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921a8aa7-21a7-48dd-bc18-a7a8210a246a admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:48.146577 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4ca42-0771-4678-9d35-de8da81f41fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:48.147119 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4ca42-0771-4678-9d35-de8da81f41fa service nova] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:48.147403 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4ca42-0771-4678-9d35-de8da81f41fa service nova] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:48.147680 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4ca42-0771-4678-9d35-de8da81f41fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:48.212793 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade4ca42-0771-4678-9d35-de8da81f41fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:48.214377 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade4ca42-0771-4678-9d35-de8da81f41fa service nova] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:48.224200 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 800/1621] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:30:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:30:48.254959 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 822/1622] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:30:47 2022] POST /identity/v3/users => generated 293 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:30:48.261163 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4ca42-0771-4678-9d35-de8da81f41fa service nova] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:48.280276 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4ca42-0771-4678-9d35-de8da81f41fa service nova] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:48.285181 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3fe3ff-a69e-400a-9821-1e41ee3c87a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:48.286503 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3fe3ff-a69e-400a-9821-1e41ee3c87a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:48.286791 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3fe3ff-a69e-400a-9821-1e41ee3c87a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:48.287187 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3fe3ff-a69e-400a-9821-1e41ee3c87a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:48.287894 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3fe3ff-a69e-400a-9821-1e41ee3c87a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:48.289509 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3fe3ff-a69e-400a-9821-1e41ee3c87a7 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:48.305863 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 801/1623] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:30:48.307223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921a8aa7-21a7-48dd-bc18-a7a8210a246a admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:48.319741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921a8aa7-21a7-48dd-bc18-a7a8210a246a admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:48.326082 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd62772-a720-4820-9807-c5da1e3b6cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:48.326827 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd62772-a720-4820-9807-c5da1e3b6cd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:48.327109 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd62772-a720-4820-9807-c5da1e3b6cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:48.327385 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd62772-a720-4820-9807-c5da1e3b6cd1 admin admin] PATH_INFO: `/v3/projects/7043d7098f5c4bfc87aa6ec6875cb13b/users/61d11b00f65549ea958510fdce2eb438/roles/7c88c815db854954b516ab97cd471647` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:48.349453 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd62772-a720-4820-9807-c5da1e3b6cd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7043d7098f5c4bfc87aa6ec6875cb13b, user_id=61d11b00f65549ea958510fdce2eb438, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:48.352263 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd62772-a720-4820-9807-c5da1e3b6cd1 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:48.491703 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 823/1624] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:30:48 2022] PUT /identity/v3/projects/7043d7098f5c4bfc87aa6ec6875cb13b/users/61d11b00f65549ea958510fdce2eb438/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:30:48.492456 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3fe3ff-a69e-400a-9821-1e41ee3c87a7 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:48.503412 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3fe3ff-a69e-400a-9821-1e41ee3c87a7 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:48.650217 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cec4ce-da48-4eac-b2d8-e7123897c766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:48.650864 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cec4ce-da48-4eac-b2d8-e7123897c766 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:48.651241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cec4ce-da48-4eac-b2d8-e7123897c766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:48.651667 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cec4ce-da48-4eac-b2d8-e7123897c766 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:48.653037 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cec4ce-da48-4eac-b2d8-e7123897c766 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:48.656900 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cec4ce-da48-4eac-b2d8-e7123897c766 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:48.681795 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 802/1625] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:30:48.685116 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd62772-a720-4820-9807-c5da1e3b6cd1 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:48.703008 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd62772-a720-4820-9807-c5da1e3b6cd1 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:48.716799 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd708a-a6c4-4f43-917c-dba57aee74d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:48.717717 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd708a-a6c4-4f43-917c-dba57aee74d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:48.718130 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd708a-a6c4-4f43-917c-dba57aee74d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:48.718493 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd708a-a6c4-4f43-917c-dba57aee74d7 admin admin] PATH_INFO: `/v3/projects/7043d7098f5c4bfc87aa6ec6875cb13b/users/61d11b00f65549ea958510fdce2eb438/roles/7c88c815db854954b516ab97cd471647` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:48.720727 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dd708a-a6c4-4f43-917c-dba57aee74d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7043d7098f5c4bfc87aa6ec6875cb13b, user_id=61d11b00f65549ea958510fdce2eb438, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:48.723876 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dd708a-a6c4-4f43-917c-dba57aee74d7 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:48.758744 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 824/1626] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:30:48 2022] PUT /identity/v3/projects/7043d7098f5c4bfc87aa6ec6875cb13b/users/61d11b00f65549ea958510fdce2eb438/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:30:48.760665 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03759fb9-cd20-4a48-aac6-e0619c7e88f8 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:48.761532 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03759fb9-cd20-4a48-aac6-e0619c7e88f8 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:48.761532 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03759fb9-cd20-4a48-aac6-e0619c7e88f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:48.808124 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-03759fb9-cd20-4a48-aac6-e0619c7e88f8 None None] MFA Rules not processed for user `61d11b00f65549ea958510fdce2eb438`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:48.809407 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-03759fb9-cd20-4a48-aac6-e0619c7e88f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:48.964756 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 803/1627] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:30:48 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:30:48.974269 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd708a-a6c4-4f43-917c-dba57aee74d7 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:49.015854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753ff9f-26ef-46c5-8e0d-32de22033f11 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:49.016231 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753ff9f-26ef-46c5-8e0d-32de22033f11 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:49.016489 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753ff9f-26ef-46c5-8e0d-32de22033f11 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:49.018215 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 804/1628] 10.222.0.40 () {54 vars in 962 bytes} [Fri Aug 5 15:30:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:30:49.041528 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d6863-8fd8-41d7-b982-74239dae1075 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:49.041528 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d6863-8fd8-41d7-b982-74239dae1075 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:49.041528 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d6863-8fd8-41d7-b982-74239dae1075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:49.058369 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:30:49.085072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-6e1d6863-8fd8-41d7-b982-74239dae1075 None None] MFA Rules not processed for user `bc0497dab1f44bb7b8196d39bf01d770`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:49.086644 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-6e1d6863-8fd8-41d7-b982-74239dae1075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:49.090029 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd708a-a6c4-4f43-917c-dba57aee74d7 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:49.094355 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b41b7a9-d175-4b34-a24e-fb7f80e79b4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:49.095055 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41b7a9-d175-4b34-a24e-fb7f80e79b4b service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:49.095331 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41b7a9-d175-4b34-a24e-fb7f80e79b4b service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:49.095670 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41b7a9-d175-4b34-a24e-fb7f80e79b4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:49.107550 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b41b7a9-d175-4b34-a24e-fb7f80e79b4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:49.109845 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b41b7a9-d175-4b34-a24e-fb7f80e79b4b service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:49.138599 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 825/1629] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:30:48 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:30:49.228494 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 805/1630] 10.222.0.40 () {58 vars in 1059 bytes} [Fri Aug 5 15:30:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:30:49.234418 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd582ae-243b-468f-9965-930e9eca1ca7 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:49.234693 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd582ae-243b-468f-9965-930e9eca1ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:49.234989 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd582ae-243b-468f-9965-930e9eca1ca7 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:49.236747 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 826/1631] 10.222.0.40 () {54 vars in 912 bytes} [Fri Aug 5 15:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:30:49.243899 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fdd01c-6e7c-4d3b-ab61-d283d731be2d None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:49.244498 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fdd01c-6e7c-4d3b-ab61-d283d731be2d None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:49.244957 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fdd01c-6e7c-4d3b-ab61-d283d731be2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:49.270654 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-73fdd01c-6e7c-4d3b-ab61-d283d731be2d None None] MFA Rules not processed for user `bc0497dab1f44bb7b8196d39bf01d770`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:49.272056 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-73fdd01c-6e7c-4d3b-ab61-d283d731be2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:49.296880 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd582ae-243b-468f-9965-930e9eca1ca7 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:49.299594 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 806/1632] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 5 15:30:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:30:49.332962 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd582ae-243b-468f-9965-930e9eca1ca7 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:49.354950 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfedd6eb-40b2-4821-b70f-72e2b5af161f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:49.356018 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfedd6eb-40b2-4821-b70f-72e2b5af161f service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:49.356539 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfedd6eb-40b2-4821-b70f-72e2b5af161f service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:49.356929 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfedd6eb-40b2-4821-b70f-72e2b5af161f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:49.383208 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfedd6eb-40b2-4821-b70f-72e2b5af161f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:49.384856 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfedd6eb-40b2-4821-b70f-72e2b5af161f service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:49.421103 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 827/1633] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:30:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:30:49.543835 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f185f-b494-473c-97c7-b0296a2ab2d8 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:49.544438 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f185f-b494-473c-97c7-b0296a2ab2d8 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:49.544806 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f185f-b494-473c-97c7-b0296a2ab2d8 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:49.546629 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 807/1634] 10.222.0.40 () {54 vars in 912 bytes} [Fri Aug 5 15:30:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:30:49.556487 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af95188-cc31-4fd7-a719-846c02b25cfe None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:49.562775 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af95188-cc31-4fd7-a719-846c02b25cfe None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:49.562775 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af95188-cc31-4fd7-a719-846c02b25cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:49.600476 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-3af95188-cc31-4fd7-a719-846c02b25cfe None None] MFA Rules not processed for user `bc0497dab1f44bb7b8196d39bf01d770`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:49.602129 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-3af95188-cc31-4fd7-a719-846c02b25cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:49.638103 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5f185f-b494-473c-97c7-b0296a2ab2d8 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:49.648924 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 828/1635] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 5 15:30:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:30:49.655460 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5f185f-b494-473c-97c7-b0296a2ab2d8 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:49.660129 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da782720-0d6e-4cfb-b2c8-f96543fb861f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:49.662405 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da782720-0d6e-4cfb-b2c8-f96543fb861f service cinder] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:49.662818 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da782720-0d6e-4cfb-b2c8-f96543fb861f service cinder] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:49.663216 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da782720-0d6e-4cfb-b2c8-f96543fb861f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:49.676606 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da782720-0d6e-4cfb-b2c8-f96543fb861f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:49.678613 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da782720-0d6e-4cfb-b2c8-f96543fb861f service cinder] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:49.708216 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 808/1636] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:30:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:30:50.147929 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af95188-cc31-4fd7-a719-846c02b25cfe None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:50.147929 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:30:50.151377 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:30:50.158531 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af95188-cc31-4fd7-a719-846c02b25cfe None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:50.158531 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b507328-5a40-43f7-80be-372dc33bfa91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:50.158531 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b507328-5a40-43f7-80be-372dc33bfa91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:50.158531 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b507328-5a40-43f7-80be-372dc33bfa91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:50.162009 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b507328-5a40-43f7-80be-372dc33bfa91 admin admin] PATH_INFO: `/v3/users/be954cca051f450f845c206c7655d841` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:50.162009 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b507328-5a40-43f7-80be-372dc33bfa91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be954cca051f450f845c206c7655d841)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:50.162009 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: WARNING py.warnings [None req-9b507328-5a40-43f7-80be-372dc33bfa91 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 15:30:50.162009 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: warnings.warn(msg) Aug 05 15:30:50.162009 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: Aug 05 15:30:50.162009 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b507328-5a40-43f7-80be-372dc33bfa91 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:50.354364 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-9b507328-5a40-43f7-80be-372dc33bfa91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be954cca051f450f845c206c7655d841'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:30:50.355204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-9b507328-5a40-43f7-80be-372dc33bfa91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be954cca051f450f845c206c7655d841'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:30:50.400014 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-9b507328-5a40-43f7-80be-372dc33bfa91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be954cca051f450f845c206c7655d841'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:30:50.518182 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-9b507328-5a40-43f7-80be-372dc33bfa91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be954cca051f450f845c206c7655d841'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:30:50.540473 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-9b507328-5a40-43f7-80be-372dc33bfa91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be954cca051f450f845c206c7655d841'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:30:50.557881 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 829/1637] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:30:49 2022] DELETE /identity/v3/users/be954cca051f450f845c206c7655d841 => generated 0 bytes in 706 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:30:50.562167 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3011e7-b2c8-4744-a431-e9d070d189fd None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:50.562655 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3011e7-b2c8-4744-a431-e9d070d189fd None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:50.562738 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3011e7-b2c8-4744-a431-e9d070d189fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:50.593501 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-fd3011e7-b2c8-4744-a431-e9d070d189fd None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:50.595552 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-fd3011e7-b2c8-4744-a431-e9d070d189fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:50.664659 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1621d-9f19-451d-bb48-4c9ffe965b4e None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:50.666525 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1621d-9f19-451d-bb48-4c9ffe965b4e None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:50.666959 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1621d-9f19-451d-bb48-4c9ffe965b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:50.687042 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 809/1638] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:30:50.719752 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3011e7-b2c8-4744-a431-e9d070d189fd None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:50.721560 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-fd3011e7-b2c8-4744-a431-e9d070d189fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:50.742113 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-c8d1621d-9f19-451d-bb48-4c9ffe965b4e None None] MFA Rules not processed for user `ec7fc07d65854cb18789d6cfe8e7d710`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:50.745885 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-c8d1621d-9f19-451d-bb48-4c9ffe965b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:50.818049 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3011e7-b2c8-4744-a431-e9d070d189fd None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:50.835886 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd039c-b761-4b79-9b12-2530ab85cdc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:50.837945 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd039c-b761-4b79-9b12-2530ab85cdc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:50.838269 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd039c-b761-4b79-9b12-2530ab85cdc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:50.840770 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd039c-b761-4b79-9b12-2530ab85cdc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:50.857641 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 830/1639] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:30:50 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:30:50.858864 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1621d-9f19-451d-bb48-4c9ffe965b4e None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:50.860611 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-c8d1621d-9f19-451d-bb48-4c9ffe965b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:50.865062 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cd039c-b761-4b79-9b12-2530ab85cdc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:50.867312 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cd039c-b761-4b79-9b12-2530ab85cdc9 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:50.878799 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1621d-9f19-451d-bb48-4c9ffe965b4e None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:50.885052 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f720ac-3573-4621-a827-7c9403b58d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:50.885954 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f720ac-3573-4621-a827-7c9403b58d76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:50.895664 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f720ac-3573-4621-a827-7c9403b58d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:50.895664 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f720ac-3573-4621-a827-7c9403b58d76 admin admin] PATH_INFO: `/v3/users/9c34950ab0aa4affb28d38ba50fa93c8` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:50.895664 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f720ac-3573-4621-a827-7c9403b58d76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c34950ab0aa4affb28d38ba50fa93c8)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:50.895664 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f720ac-3573-4621-a827-7c9403b58d76 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:50.896015 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 810/1640] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:30:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:30:50.897468 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd039c-b761-4b79-9b12-2530ab85cdc9 service neutron] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:50.902869 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-12cd039c-b761-4b79-9b12-2530ab85cdc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:50.922597 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd039c-b761-4b79-9b12-2530ab85cdc9 service neutron] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:50.928603 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98318174-2f94-474e-90a1-a923f8678065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:50.929654 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98318174-2f94-474e-90a1-a923f8678065 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:50.930222 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98318174-2f94-474e-90a1-a923f8678065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:50.930734 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98318174-2f94-474e-90a1-a923f8678065 admin admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:50.931646 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98318174-2f94-474e-90a1-a923f8678065 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:50.933921 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98318174-2f94-474e-90a1-a923f8678065 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:51.081108 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 811/1641] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:30:50 2022] POST /identity/v3/projects => generated 405 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:30:51.086367 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98318174-2f94-474e-90a1-a923f8678065 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:51.114569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-36f720ac-3573-4621-a827-7c9403b58d76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c34950ab0aa4affb28d38ba50fa93c8'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:30:51.115424 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-36f720ac-3573-4621-a827-7c9403b58d76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c34950ab0aa4affb28d38ba50fa93c8'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:30:51.137791 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:30:51.143424 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-36f720ac-3573-4621-a827-7c9403b58d76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c34950ab0aa4affb28d38ba50fa93c8'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:30:51.170032 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98318174-2f94-474e-90a1-a923f8678065 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:51.183846 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2791eb-53cb-49a9-8251-0f3f8dfd46f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:51.184538 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2791eb-53cb-49a9-8251-0f3f8dfd46f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:51.184833 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2791eb-53cb-49a9-8251-0f3f8dfd46f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:51.185307 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2791eb-53cb-49a9-8251-0f3f8dfd46f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:51.186210 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2791eb-53cb-49a9-8251-0f3f8dfd46f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:51.197584 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2791eb-53cb-49a9-8251-0f3f8dfd46f2 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:51.224730 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-36f720ac-3573-4621-a827-7c9403b58d76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c34950ab0aa4affb28d38ba50fa93c8'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:30:51.234181 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-36f720ac-3573-4621-a827-7c9403b58d76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c34950ab0aa4affb28d38ba50fa93c8'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:30:51.248673 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 831/1642] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:30:50 2022] DELETE /identity/v3/users/9c34950ab0aa4affb28d38ba50fa93c8 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:30:51.263933 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d7e7f-df8c-40c0-bcb8-95026b90df4e None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:51.264535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d7e7f-df8c-40c0-bcb8-95026b90df4e None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:51.265287 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d7e7f-df8c-40c0-bcb8-95026b90df4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:51.312637 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-a70d7e7f-df8c-40c0-bcb8-95026b90df4e None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:51.319309 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-a70d7e7f-df8c-40c0-bcb8-95026b90df4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:51.365493 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 812/1643] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:30:51 2022] POST /identity/v3/users => generated 309 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:30:51.366764 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2791eb-53cb-49a9-8251-0f3f8dfd46f2 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:51.368922 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-aa2791eb-53cb-49a9-8251-0f3f8dfd46f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:51.424127 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2791eb-53cb-49a9-8251-0f3f8dfd46f2 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:51.460104 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:30:51.465810 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 832/1644] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:30:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:30:51.521079 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70d7e7f-df8c-40c0-bcb8-95026b90df4e None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:51.524396 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-a70d7e7f-df8c-40c0-bcb8-95026b90df4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:51.575165 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96461b-4c97-4ae1-8ede-4fc324f361bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:51.577871 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96461b-4c97-4ae1-8ede-4fc324f361bc admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:51.578165 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96461b-4c97-4ae1-8ede-4fc324f361bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:51.578450 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96461b-4c97-4ae1-8ede-4fc324f361bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:51.580932 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a96461b-4c97-4ae1-8ede-4fc324f361bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:51.588756 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a96461b-4c97-4ae1-8ede-4fc324f361bc admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:51.620961 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 813/1645] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:30:51.625876 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96461b-4c97-4ae1-8ede-4fc324f361bc admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:51.655387 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96461b-4c97-4ae1-8ede-4fc324f361bc admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:51.674744 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4742e0-a956-4fba-a450-a85b032d23d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:51.679397 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4742e0-a956-4fba-a450-a85b032d23d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:51.680608 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4742e0-a956-4fba-a450-a85b032d23d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:51.685210 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4742e0-a956-4fba-a450-a85b032d23d1 admin admin] PATH_INFO: `/v3/projects/05b57b1a5cde4522a3517babee3cf463/users/8b2513d0c46c4a2b9585ce2ad7d705de/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:51.709969 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70d7e7f-df8c-40c0-bcb8-95026b90df4e None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:51.730814 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdcc1b0-8c65-4038-808c-6ed187633051 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:51.732669 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdcc1b0-8c65-4038-808c-6ed187633051 service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:51.735156 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdcc1b0-8c65-4038-808c-6ed187633051 service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:51.735460 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdcc1b0-8c65-4038-808c-6ed187633051 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:51.771607 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4742e0-a956-4fba-a450-a85b032d23d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05b57b1a5cde4522a3517babee3cf463, user_id=8b2513d0c46c4a2b9585ce2ad7d705de, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:51.778801 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4742e0-a956-4fba-a450-a85b032d23d1 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:51.786290 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdcc1b0-8c65-4038-808c-6ed187633051 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:51.793509 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdcc1b0-8c65-4038-808c-6ed187633051 service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:51.812582 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 833/1646] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:30:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:30:51.896285 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdcc1b0-8c65-4038-808c-6ed187633051 service neutron] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:51.904849 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 814/1647] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:30:51 2022] PUT /identity/v3/projects/05b57b1a5cde4522a3517babee3cf463/users/8b2513d0c46c4a2b9585ce2ad7d705de/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:30:51.913174 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdcc1b0-8c65-4038-808c-6ed187633051 service neutron] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:52.057499 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2863ad6d-b9d2-4696-82e7-f7c9ed4285b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:52.060979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863ad6d-b9d2-4696-82e7-f7c9ed4285b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:52.064704 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863ad6d-b9d2-4696-82e7-f7c9ed4285b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:52.068662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863ad6d-b9d2-4696-82e7-f7c9ed4285b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:52.068662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2863ad6d-b9d2-4696-82e7-f7c9ed4285b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:52.068874 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2863ad6d-b9d2-4696-82e7-f7c9ed4285b7 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:52.108501 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 834/1648] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:30:52.124631 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4742e0-a956-4fba-a450-a85b032d23d1 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:52.146288 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4742e0-a956-4fba-a450-a85b032d23d1 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:52.167374 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab325040-9670-40e7-aa86-a259d385bc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:52.168090 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab325040-9670-40e7-aa86-a259d385bc96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:52.168393 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab325040-9670-40e7-aa86-a259d385bc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:52.168671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab325040-9670-40e7-aa86-a259d385bc96 admin admin] PATH_INFO: `/v3/projects/05b57b1a5cde4522a3517babee3cf463/users/8b2513d0c46c4a2b9585ce2ad7d705de/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:52.178122 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab325040-9670-40e7-aa86-a259d385bc96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05b57b1a5cde4522a3517babee3cf463, user_id=8b2513d0c46c4a2b9585ce2ad7d705de, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:52.184326 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab325040-9670-40e7-aa86-a259d385bc96 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:52.231512 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 815/1649] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:30:52 2022] PUT /identity/v3/projects/05b57b1a5cde4522a3517babee3cf463/users/8b2513d0c46c4a2b9585ce2ad7d705de/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:30:52.485107 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2863ad6d-b9d2-4696-82e7-f7c9ed4285b7 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:52.486639 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-2863ad6d-b9d2-4696-82e7-f7c9ed4285b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:52.513844 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2863ad6d-b9d2-4696-82e7-f7c9ed4285b7 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:52.660576 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a266096-d7e8-4bd1-b312-8df703bf47ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:52.661245 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a266096-d7e8-4bd1-b312-8df703bf47ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:52.661528 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a266096-d7e8-4bd1-b312-8df703bf47ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:52.661807 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a266096-d7e8-4bd1-b312-8df703bf47ab admin admin] PATH_INFO: `/v3/projects/521158d79b45465298460c240f399055` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:52.663539 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a266096-d7e8-4bd1-b312-8df703bf47ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=521158d79b45465298460c240f399055)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:52.669159 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a266096-d7e8-4bd1-b312-8df703bf47ab admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:52.846908 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-0a266096-d7e8-4bd1-b312-8df703bf47ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '521158d79b45465298460c240f399055', 'request_id': 'req-0a266096-d7e8-4bd1-b312-8df703bf47ab'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:30:52.874818 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-0a266096-d7e8-4bd1-b312-8df703bf47ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '521158d79b45465298460c240f399055', 'request_id': 'req-0a266096-d7e8-4bd1-b312-8df703bf47ab'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:30:52.887106 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-0a266096-d7e8-4bd1-b312-8df703bf47ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '521158d79b45465298460c240f399055', 'request_id': 'req-0a266096-d7e8-4bd1-b312-8df703bf47ab'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:30:53.034874 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-0a266096-d7e8-4bd1-b312-8df703bf47ab admin admin] The token cache is being invalidate because project 521158d79b45465298460c240f399055 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:30:53.035225 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-0a266096-d7e8-4bd1-b312-8df703bf47ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:30:53.035950 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-0a266096-d7e8-4bd1-b312-8df703bf47ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:30:53.039872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 835/1650] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:30:52 2022] DELETE /identity/v3/projects/521158d79b45465298460c240f399055 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:30:53.044767 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111157d-d3bf-4c1a-a759-9a4b1736d8b8 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:53.045244 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111157d-d3bf-4c1a-a759-9a4b1736d8b8 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:53.052963 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111157d-d3bf-4c1a-a759-9a4b1736d8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:53.104485 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-b111157d-d3bf-4c1a-a759-9a4b1736d8b8 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:53.105734 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-b111157d-d3bf-4c1a-a759-9a4b1736d8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:53.251384 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 816/1651] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:30:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:30:53.258833 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e9d89-0f53-49b5-be2e-252032829f63 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:53.259146 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e9d89-0f53-49b5-be2e-252032829f63 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:53.259430 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e9d89-0f53-49b5-be2e-252032829f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:53.277365 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:30:53.315952 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-a99e9d89-0f53-49b5-be2e-252032829f63 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:53.322290 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-a99e9d89-0f53-49b5-be2e-252032829f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:53.354537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 836/1652] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:30:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:30:53.355938 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111157d-d3bf-4c1a-a759-9a4b1736d8b8 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:53.375299 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99e9d89-0f53-49b5-be2e-252032829f63 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:53.377138 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-a99e9d89-0f53-49b5-be2e-252032829f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:53.400907 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111157d-d3bf-4c1a-a759-9a4b1736d8b8 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:53.406722 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99e9d89-0f53-49b5-be2e-252032829f63 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:53.420879 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e4ec96-e812-4742-8404-b139c8ca4163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:53.424084 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e4ec96-e812-4742-8404-b139c8ca4163 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:53.424407 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ac516-8dac-4f87-bc30-72ce1853b05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:53.425297 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ac516-8dac-4f87-bc30-72ce1853b05e admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:53.426290 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e4ec96-e812-4742-8404-b139c8ca4163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:53.426592 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ac516-8dac-4f87-bc30-72ce1853b05e admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:53.426688 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e4ec96-e812-4742-8404-b139c8ca4163 admin admin] PATH_INFO: `/v3/projects/2ff2e00771274e4ea0ca0198dabb11a1` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:53.427195 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ac516-8dac-4f87-bc30-72ce1853b05e admin admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:53.429367 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ac516-8dac-4f87-bc30-72ce1853b05e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:30:53.429474 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e4ec96-e812-4742-8404-b139c8ca4163 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ff2e00771274e4ea0ca0198dabb11a1)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:53.430229 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ac516-8dac-4f87-bc30-72ce1853b05e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:53.435436 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ac516-8dac-4f87-bc30-72ce1853b05e admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:53.437522 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e4ec96-e812-4742-8404-b139c8ca4163 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:53.479075 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 817/1653] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:30:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:30:53.479075 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ac516-8dac-4f87-bc30-72ce1853b05e admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:53.528942 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ac516-8dac-4f87-bc30-72ce1853b05e admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:53.543606 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765d800-86e1-41bd-98c3-0b69dd3436fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:53.544415 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765d800-86e1-41bd-98c3-0b69dd3436fb admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:53.546694 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765d800-86e1-41bd-98c3-0b69dd3436fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:53.546694 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765d800-86e1-41bd-98c3-0b69dd3436fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:53.548846 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5765d800-86e1-41bd-98c3-0b69dd3436fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:53.552939 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5765d800-86e1-41bd-98c3-0b69dd3436fb admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:53.574833 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:30:53.626756 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-66e4ec96-e812-4742-8404-b139c8ca4163 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ff2e00771274e4ea0ca0198dabb11a1', 'request_id': 'req-66e4ec96-e812-4742-8404-b139c8ca4163'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:30:53.650829 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-66e4ec96-e812-4742-8404-b139c8ca4163 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ff2e00771274e4ea0ca0198dabb11a1', 'request_id': 'req-66e4ec96-e812-4742-8404-b139c8ca4163'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:30:53.678511 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-66e4ec96-e812-4742-8404-b139c8ca4163 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ff2e00771274e4ea0ca0198dabb11a1', 'request_id': 'req-66e4ec96-e812-4742-8404-b139c8ca4163'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:30:53.754442 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 818/1654] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:30:53 2022] POST /identity/v3/projects => generated 379 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:30:53.754537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-66e4ec96-e812-4742-8404-b139c8ca4163 admin admin] The token cache is being invalidate because project 2ff2e00771274e4ea0ca0198dabb11a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:30:53.754537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-66e4ec96-e812-4742-8404-b139c8ca4163 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:30:53.754537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-66e4ec96-e812-4742-8404-b139c8ca4163 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:30:53.755781 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765d800-86e1-41bd-98c3-0b69dd3436fb admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:53.758597 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-5765d800-86e1-41bd-98c3-0b69dd3436fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:53.763113 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 837/1655] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:30:53 2022] DELETE /identity/v3/projects/2ff2e00771274e4ea0ca0198dabb11a1 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:30:53.768571 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e4ec96-e812-4742-8404-b139c8ca4163 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:53.774623 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-66e4ec96-e812-4742-8404-b139c8ca4163 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:53.996710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e4ec96-e812-4742-8404-b139c8ca4163 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:54.002187 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4fe955-d713-4f12-b425-b9cbaaa2c38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:54.003016 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4fe955-d713-4f12-b425-b9cbaaa2c38a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:54.004984 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4fe955-d713-4f12-b425-b9cbaaa2c38a admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:54.005449 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4fe955-d713-4f12-b425-b9cbaaa2c38a admin admin] PATH_INFO: `/v3/users/61d11b00f65549ea958510fdce2eb438` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:54.012406 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4fe955-d713-4f12-b425-b9cbaaa2c38a admin admin] RBAC: Authorizing `identity:delete_user(user_id=61d11b00f65549ea958510fdce2eb438)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:54.012406 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4fe955-d713-4f12-b425-b9cbaaa2c38a admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:54.013925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765d800-86e1-41bd-98c3-0b69dd3436fb admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:54.042307 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2920d517-d235-4518-b38b-990458ab3c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:54.042307 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920d517-d235-4518-b38b-990458ab3c95 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:54.042607 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920d517-d235-4518-b38b-990458ab3c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:54.046869 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920d517-d235-4518-b38b-990458ab3c95 admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:54.051978 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2920d517-d235-4518-b38b-990458ab3c95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:54.065561 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2920d517-d235-4518-b38b-990458ab3c95 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:54.215527 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 819/1656] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:30:53 2022] POST /identity/v3/users => generated 290 bytes in 460 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:30:54.220816 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2920d517-d235-4518-b38b-990458ab3c95 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:54.258951 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2920d517-d235-4518-b38b-990458ab3c95 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:54.262665 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-8b4fe955-d713-4f12-b425-b9cbaaa2c38a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61d11b00f65549ea958510fdce2eb438'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:30:54.264345 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-8b4fe955-d713-4f12-b425-b9cbaaa2c38a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61d11b00f65549ea958510fdce2eb438'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:30:54.288907 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84730fa9-9b22-4b02-bdad-029cf96b4dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:54.289823 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84730fa9-9b22-4b02-bdad-029cf96b4dbd admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:54.290244 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84730fa9-9b22-4b02-bdad-029cf96b4dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:54.290658 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84730fa9-9b22-4b02-bdad-029cf96b4dbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:54.291804 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84730fa9-9b22-4b02-bdad-029cf96b4dbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:54.293715 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84730fa9-9b22-4b02-bdad-029cf96b4dbd admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:54.311008 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-8b4fe955-d713-4f12-b425-b9cbaaa2c38a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61d11b00f65549ea958510fdce2eb438'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:30:54.332164 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 820/1657] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:30:54 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:30:54.333266 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84730fa9-9b22-4b02-bdad-029cf96b4dbd admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:54.347835 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-84730fa9-9b22-4b02-bdad-029cf96b4dbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:54.367330 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-8b4fe955-d713-4f12-b425-b9cbaaa2c38a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61d11b00f65549ea958510fdce2eb438'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:30:54.381628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-8b4fe955-d713-4f12-b425-b9cbaaa2c38a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61d11b00f65549ea958510fdce2eb438'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:30:54.396235 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 838/1658] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:30:53 2022] DELETE /identity/v3/users/61d11b00f65549ea958510fdce2eb438 => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:30:54.398946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84730fa9-9b22-4b02-bdad-029cf96b4dbd admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:54.550078 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:30:54.590928 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99e1c62-cce1-4c64-9de5-ad2043030c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:54.591796 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e1c62-cce1-4c64-9de5-ad2043030c6d admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:54.592276 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e1c62-cce1-4c64-9de5-ad2043030c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:54.593553 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e1c62-cce1-4c64-9de5-ad2043030c6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:54.600368 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99e1c62-cce1-4c64-9de5-ad2043030c6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:54.608810 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99e1c62-cce1-4c64-9de5-ad2043030c6d admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:54.627125 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 821/1659] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:30:54 2022] GET /identity/v3/roles => generated 1420 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:30:54.631588 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4fe955-d713-4f12-b425-b9cbaaa2c38a admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:54.646463 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4fe955-d713-4f12-b425-b9cbaaa2c38a admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:54.664311 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418698b-584a-4a63-96a1-3707b7fc4307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:54.665527 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418698b-584a-4a63-96a1-3707b7fc4307 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:54.665978 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418698b-584a-4a63-96a1-3707b7fc4307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:54.666394 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418698b-584a-4a63-96a1-3707b7fc4307 admin admin] PATH_INFO: `/v3/projects/f54498df9fc64213b1b50cfc558e3502/users/bb797da54e734a93be8524b43b8c345f/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:54.709150 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c418698b-584a-4a63-96a1-3707b7fc4307 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f54498df9fc64213b1b50cfc558e3502, user_id=bb797da54e734a93be8524b43b8c345f, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:54.715138 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c418698b-584a-4a63-96a1-3707b7fc4307 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:54.842768 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 839/1660] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:30:54 2022] PUT /identity/v3/projects/f54498df9fc64213b1b50cfc558e3502/users/bb797da54e734a93be8524b43b8c345f/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:30:54.853864 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695aef1-4fc7-468e-8269-64c50346b81a None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:54.854155 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695aef1-4fc7-468e-8269-64c50346b81a None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:54.854434 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695aef1-4fc7-468e-8269-64c50346b81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:54.921474 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-b695aef1-4fc7-468e-8269-64c50346b81a None None] MFA Rules not processed for user `bb797da54e734a93be8524b43b8c345f`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:54.922883 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-b695aef1-4fc7-468e-8269-64c50346b81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:55.057331 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:30:55.075831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 822/1661] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:30:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:30:55.076960 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418698b-584a-4a63-96a1-3707b7fc4307 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:55.261708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418698b-584a-4a63-96a1-3707b7fc4307 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:55.266321 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aba670-cb19-48b5-b337-1779222ac4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:55.267949 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aba670-cb19-48b5-b337-1779222ac4dc admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:55.267949 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aba670-cb19-48b5-b337-1779222ac4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:55.267949 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aba670-cb19-48b5-b337-1779222ac4dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:55.272776 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aba670-cb19-48b5-b337-1779222ac4dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:55.274820 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aba670-cb19-48b5-b337-1779222ac4dc admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:55.448354 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b695aef1-4fc7-468e-8269-64c50346b81a None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:55.465530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 840/1662] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:30:55 2022] POST /identity/v3/projects => generated 379 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:30:55.468938 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b695aef1-4fc7-468e-8269-64c50346b81a None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:55.553589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9f9308-71fc-4d91-adc3-dd6a53a70572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:55.555110 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f9308-71fc-4d91-adc3-dd6a53a70572 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:55.555110 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f9308-71fc-4d91-adc3-dd6a53a70572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:55.555110 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f9308-71fc-4d91-adc3-dd6a53a70572 admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:55.555484 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9f9308-71fc-4d91-adc3-dd6a53a70572 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:55.557560 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9f9308-71fc-4d91-adc3-dd6a53a70572 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:55.663029 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 823/1663] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:30:55 2022] POST /identity/v3/users => generated 296 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:30:55.667447 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aba670-cb19-48b5-b337-1779222ac4dc admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:55.683526 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aba670-cb19-48b5-b337-1779222ac4dc admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:55.687550 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b158a1-49b8-4421-8a98-28b54b1e4f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:55.688535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b158a1-49b8-4421-8a98-28b54b1e4f4f admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:55.688535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b158a1-49b8-4421-8a98-28b54b1e4f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:55.688740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b158a1-49b8-4421-8a98-28b54b1e4f4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:55.689415 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b158a1-49b8-4421-8a98-28b54b1e4f4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:55.691741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b158a1-49b8-4421-8a98-28b54b1e4f4f admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:55.711490 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 841/1664] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:30:55.712047 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9f9308-71fc-4d91-adc3-dd6a53a70572 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:55.725291 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9f9308-71fc-4d91-adc3-dd6a53a70572 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:55.743141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76650c35-d9f8-437a-a6be-721aba4c4b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:55.744928 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76650c35-d9f8-437a-a6be-721aba4c4b9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:55.744928 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76650c35-d9f8-437a-a6be-721aba4c4b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:55.744928 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76650c35-d9f8-437a-a6be-721aba4c4b9d admin admin] PATH_INFO: `/v3/projects/a4715fb4480b4119b8212f2f4b0c6063/users/fd1776073693456d89a754852ef1c804/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:55.798842 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76650c35-d9f8-437a-a6be-721aba4c4b9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4715fb4480b4119b8212f2f4b0c6063, user_id=fd1776073693456d89a754852ef1c804, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:55.808375 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76650c35-d9f8-437a-a6be-721aba4c4b9d admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:55.920717 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 824/1665] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:30:55 2022] PUT /identity/v3/projects/a4715fb4480b4119b8212f2f4b0c6063/users/fd1776073693456d89a754852ef1c804/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:30:55.923206 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b158a1-49b8-4421-8a98-28b54b1e4f4f admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:55.943335 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b158a1-49b8-4421-8a98-28b54b1e4f4f admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:56.025887 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:30:56.093828 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb966c-e895-41df-9ebe-83d78187aa51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:56.098290 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb966c-e895-41df-9ebe-83d78187aa51 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:56.102535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb966c-e895-41df-9ebe-83d78187aa51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:56.104896 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb966c-e895-41df-9ebe-83d78187aa51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:56.105782 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eb966c-e895-41df-9ebe-83d78187aa51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:56.107558 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eb966c-e895-41df-9ebe-83d78187aa51 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:56.198156 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 842/1666] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:30:56.201384 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76650c35-d9f8-437a-a6be-721aba4c4b9d admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:56.216573 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76650c35-d9f8-437a-a6be-721aba4c4b9d admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:56.237928 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d4602-a4f5-4f03-a964-774fe89dacdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:56.239269 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d4602-a4f5-4f03-a964-774fe89dacdf admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:56.239770 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d4602-a4f5-4f03-a964-774fe89dacdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:56.240204 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d4602-a4f5-4f03-a964-774fe89dacdf admin admin] PATH_INFO: `/v3/projects/a4715fb4480b4119b8212f2f4b0c6063/users/fd1776073693456d89a754852ef1c804/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:56.246089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6d4602-a4f5-4f03-a964-774fe89dacdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4715fb4480b4119b8212f2f4b0c6063, user_id=fd1776073693456d89a754852ef1c804, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:56.259013 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6d4602-a4f5-4f03-a964-774fe89dacdf admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:56.294315 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 825/1667] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:30:56 2022] PUT /identity/v3/projects/a4715fb4480b4119b8212f2f4b0c6063/users/fd1776073693456d89a754852ef1c804/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:30:56.306241 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78a05f-5a20-48bc-b905-435035b7276a None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:56.311646 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78a05f-5a20-48bc-b905-435035b7276a None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:56.312237 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78a05f-5a20-48bc-b905-435035b7276a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:56.414225 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-cf78a05f-5a20-48bc-b905-435035b7276a None None] MFA Rules not processed for user `fd1776073693456d89a754852ef1c804`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:56.415667 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-cf78a05f-5a20-48bc-b905-435035b7276a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:56.510033 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 843/1668] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:30:56 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:30:56.522549 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d4602-a4f5-4f03-a964-774fe89dacdf admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:56.524685 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-6c6d4602-a4f5-4f03-a964-774fe89dacdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:56.680010 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d4602-a4f5-4f03-a964-774fe89dacdf admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:56.689498 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f437d-3d33-4927-85d5-416379b090da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:56.695504 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f437d-3d33-4927-85d5-416379b090da service cinder] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:56.695504 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f437d-3d33-4927-85d5-416379b090da service cinder] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:56.695504 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f437d-3d33-4927-85d5-416379b090da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:56.717146 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f437d-3d33-4927-85d5-416379b090da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:56.717146 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f437d-3d33-4927-85d5-416379b090da service cinder] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:56.746330 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 826/1669] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:30:56 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:30:56.935360 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf78a05f-5a20-48bc-b905-435035b7276a None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:56.935360 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-cf78a05f-5a20-48bc-b905-435035b7276a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:57.007316 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d82d9-d02b-4964-b90c-bb735cb78409 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:57.007604 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d82d9-d02b-4964-b90c-bb735cb78409 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:57.007880 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d82d9-d02b-4964-b90c-bb735cb78409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:57.105119 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-d38d82d9-d02b-4964-b90c-bb735cb78409 None None] MFA Rules not processed for user `8b2513d0c46c4a2b9585ce2ad7d705de`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:57.106592 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-d38d82d9-d02b-4964-b90c-bb735cb78409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:57.126259 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf78a05f-5a20-48bc-b905-435035b7276a None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:57.141872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eac4d0-b7ce-4fc2-b672-1aa3835a1d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:57.142687 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eac4d0-b7ce-4fc2-b672-1aa3835a1d57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:57.143089 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eac4d0-b7ce-4fc2-b672-1aa3835a1d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:57.143477 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eac4d0-b7ce-4fc2-b672-1aa3835a1d57 admin admin] PATH_INFO: `/v3/projects/7043d7098f5c4bfc87aa6ec6875cb13b` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:57.145407 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eac4d0-b7ce-4fc2-b672-1aa3835a1d57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7043d7098f5c4bfc87aa6ec6875cb13b)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:57.149575 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eac4d0-b7ce-4fc2-b672-1aa3835a1d57 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:57.179960 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:30:57.201596 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:30:57.304673 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 827/1670] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:30:57 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:30:57.305922 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d82d9-d02b-4964-b90c-bb735cb78409 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:57.311548 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-f1eac4d0-b7ce-4fc2-b672-1aa3835a1d57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7043d7098f5c4bfc87aa6ec6875cb13b', 'request_id': 'req-f1eac4d0-b7ce-4fc2-b672-1aa3835a1d57'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:30:57.325766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-f1eac4d0-b7ce-4fc2-b672-1aa3835a1d57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7043d7098f5c4bfc87aa6ec6875cb13b', 'request_id': 'req-f1eac4d0-b7ce-4fc2-b672-1aa3835a1d57'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:30:57.331518 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d82d9-d02b-4964-b90c-bb735cb78409 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:57.346876 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-f1eac4d0-b7ce-4fc2-b672-1aa3835a1d57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7043d7098f5c4bfc87aa6ec6875cb13b', 'request_id': 'req-f1eac4d0-b7ce-4fc2-b672-1aa3835a1d57'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:30:57.452544 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-f1eac4d0-b7ce-4fc2-b672-1aa3835a1d57 admin admin] The token cache is being invalidate because project 7043d7098f5c4bfc87aa6ec6875cb13b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:30:57.453036 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-f1eac4d0-b7ce-4fc2-b672-1aa3835a1d57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:30:57.454268 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-f1eac4d0-b7ce-4fc2-b672-1aa3835a1d57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:30:57.461798 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 844/1671] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:30:56 2022] DELETE /identity/v3/projects/7043d7098f5c4bfc87aa6ec6875cb13b => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:30:57.464783 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956088c-e60f-41d1-b9b5-65471eb4d1b5 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:57.465183 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956088c-e60f-41d1-b9b5-65471eb4d1b5 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:57.465576 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956088c-e60f-41d1-b9b5-65471eb4d1b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:57.473634 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224fb1cc-dc7c-4ed4-a58a-b0048cadd564 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:57.474584 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224fb1cc-dc7c-4ed4-a58a-b0048cadd564 service cinder] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:57.475980 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224fb1cc-dc7c-4ed4-a58a-b0048cadd564 service cinder] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:57.476447 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224fb1cc-dc7c-4ed4-a58a-b0048cadd564 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:57.482586 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-224fb1cc-dc7c-4ed4-a58a-b0048cadd564 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:57.509145 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-9956088c-e60f-41d1-b9b5-65471eb4d1b5 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:57.512412 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-9956088c-e60f-41d1-b9b5-65471eb4d1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:57.677655 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 845/1672] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:30:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:30:57.680476 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81e170-28c4-494f-8323-b1641324758f None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:57.680842 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81e170-28c4-494f-8323-b1641324758f None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:57.681234 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81e170-28c4-494f-8323-b1641324758f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:57.693994 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224fb1cc-dc7c-4ed4-a58a-b0048cadd564 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:57.703427 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224fb1cc-dc7c-4ed4-a58a-b0048cadd564 service cinder] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:57.722136 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-fb81e170-28c4-494f-8323-b1641324758f None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:57.724287 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-fb81e170-28c4-494f-8323-b1641324758f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:57.736157 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 828/1673] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:30:57 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:30:57.756255 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 846/1674] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:30:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:30:57.756473 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224fb1cc-dc7c-4ed4-a58a-b0048cadd564 service cinder] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:57.775085 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224fb1cc-dc7c-4ed4-a58a-b0048cadd564 service cinder] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:57.789295 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb2ddad-808f-4a2e-8d5e-90a12387e536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:57.791011 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2ddad-808f-4a2e-8d5e-90a12387e536 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:57.791306 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2ddad-808f-4a2e-8d5e-90a12387e536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:57.793620 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2ddad-808f-4a2e-8d5e-90a12387e536 admin admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:57.794457 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb2ddad-808f-4a2e-8d5e-90a12387e536 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:30:57.794788 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb2ddad-808f-4a2e-8d5e-90a12387e536 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:57.801363 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb2ddad-808f-4a2e-8d5e-90a12387e536 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:57.863668 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 829/1675] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:30:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:30:57.867705 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81e170-28c4-494f-8323-b1641324758f None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:57.887796 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81e170-28c4-494f-8323-b1641324758f None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:57.919873 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239efc9f-89da-4495-8378-fc48fd86ad39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:57.920740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239efc9f-89da-4495-8378-fc48fd86ad39 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:57.921101 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239efc9f-89da-4495-8378-fc48fd86ad39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:57.921482 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239efc9f-89da-4495-8378-fc48fd86ad39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:57.923670 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239efc9f-89da-4495-8378-fc48fd86ad39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:57.927855 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239efc9f-89da-4495-8378-fc48fd86ad39 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:58.079938 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 847/1676] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:30:57 2022] POST /identity/v3/projects => generated 365 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:30:58.084269 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb2ddad-808f-4a2e-8d5e-90a12387e536 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:58.119497 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb2ddad-808f-4a2e-8d5e-90a12387e536 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:58.286240 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d339832-25dc-45f9-9c87-21900c791857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:58.287501 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d339832-25dc-45f9-9c87-21900c791857 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:58.288429 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d339832-25dc-45f9-9c87-21900c791857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:58.290628 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d339832-25dc-45f9-9c87-21900c791857 admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:58.291529 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d339832-25dc-45f9-9c87-21900c791857 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:58.295795 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d339832-25dc-45f9-9c87-21900c791857 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:58.452429 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 830/1677] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:30:58 2022] POST /identity/v3/users => generated 283 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:30:58.453457 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239efc9f-89da-4495-8378-fc48fd86ad39 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:58.471859 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239efc9f-89da-4495-8378-fc48fd86ad39 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:58.484787 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d113426-ea88-4d74-95b5-28e917ecf275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:58.487347 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d113426-ea88-4d74-95b5-28e917ecf275 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:58.487642 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d113426-ea88-4d74-95b5-28e917ecf275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:58.488719 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d113426-ea88-4d74-95b5-28e917ecf275 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:58.490762 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d113426-ea88-4d74-95b5-28e917ecf275 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:58.497961 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d113426-ea88-4d74-95b5-28e917ecf275 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:58.538924 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 848/1678] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 15:30:58.542820 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d339832-25dc-45f9-9c87-21900c791857 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:58.560335 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d339832-25dc-45f9-9c87-21900c791857 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:58.568600 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0ba11d-7f50-4c64-903d-0157e28adb79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:58.569254 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ba11d-7f50-4c64-903d-0157e28adb79 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:58.569542 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ba11d-7f50-4c64-903d-0157e28adb79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:58.569837 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ba11d-7f50-4c64-903d-0157e28adb79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:58.570542 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0ba11d-7f50-4c64-903d-0157e28adb79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:58.578562 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:30:58.581167 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0ba11d-7f50-4c64-903d-0157e28adb79 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:58.610848 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d113426-ea88-4d74-95b5-28e917ecf275 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:58.616565 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 831/1679] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:30:58.623397 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d113426-ea88-4d74-95b5-28e917ecf275 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:58.643261 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4637b2bc-48e4-42d3-88e2-45affa09a99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:58.643261 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637b2bc-48e4-42d3-88e2-45affa09a99b admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:58.643261 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637b2bc-48e4-42d3-88e2-45affa09a99b admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:58.643261 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637b2bc-48e4-42d3-88e2-45affa09a99b admin admin] PATH_INFO: `/v3/projects/3c58b93a5db6487cb0b3061ab3f0aa1c/users/3e51c27465d34d14b022425b95ad8734/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:58.655574 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a9757-875d-4362-ad60-b033b5f964ed None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:58.655874 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a9757-875d-4362-ad60-b033b5f964ed None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:58.658994 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a9757-875d-4362-ad60-b033b5f964ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:58.680297 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:30:58.705115 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4637b2bc-48e4-42d3-88e2-45affa09a99b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c58b93a5db6487cb0b3061ab3f0aa1c, user_id=3e51c27465d34d14b022425b95ad8734, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:58.717165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4637b2bc-48e4-42d3-88e2-45affa09a99b admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:58.732271 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-b64a9757-875d-4362-ad60-b033b5f964ed None None] MFA Rules not processed for user `cb055110c77543c1861dfd8c155d7582`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:58.733561 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-b64a9757-875d-4362-ad60-b033b5f964ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:58.847132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 849/1680] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:30:58 2022] PUT /identity/v3/projects/3c58b93a5db6487cb0b3061ab3f0aa1c/users/3e51c27465d34d14b022425b95ad8734/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:30:58.858419 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e195a-a6db-404c-9aa6-d2eb7e81d5a2 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:58.858722 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e195a-a6db-404c-9aa6-d2eb7e81d5a2 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:58.859011 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e195a-a6db-404c-9aa6-d2eb7e81d5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:58.928709 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-cf3e195a-a6db-404c-9aa6-d2eb7e81d5a2 None None] MFA Rules not processed for user `3e51c27465d34d14b022425b95ad8734`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:58.929968 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-cf3e195a-a6db-404c-9aa6-d2eb7e81d5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:58.943667 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 832/1681] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:30:58 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:30:58.958924 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f2b3e-a41b-4916-87a5-12c9538ec624 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:58.958924 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f2b3e-a41b-4916-87a5-12c9538ec624 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:58.958924 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f2b3e-a41b-4916-87a5-12c9538ec624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:59.002839 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-566f2b3e-a41b-4916-87a5-12c9538ec624 None None] MFA Rules not processed for user `cb055110c77543c1861dfd8c155d7582`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:30:59.004771 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-566f2b3e-a41b-4916-87a5-12c9538ec624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:59.116383 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 850/1682] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:30:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:30:59.121328 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e195a-a6db-404c-9aa6-d2eb7e81d5a2 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:59.147717 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 833/1683] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:30:58 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:30:59.157517 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f2b3e-a41b-4916-87a5-12c9538ec624 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:59.158965 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-566f2b3e-a41b-4916-87a5-12c9538ec624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:30:59.293814 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e195a-a6db-404c-9aa6-d2eb7e81d5a2 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:59.309984 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef54f794-13f7-4b96-a57e-06ddbf9cb133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:59.310948 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54f794-13f7-4b96-a57e-06ddbf9cb133 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:59.311500 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54f794-13f7-4b96-a57e-06ddbf9cb133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:59.312725 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54f794-13f7-4b96-a57e-06ddbf9cb133 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:59.313623 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef54f794-13f7-4b96-a57e-06ddbf9cb133 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:59.313877 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f2b3e-a41b-4916-87a5-12c9538ec624 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:59.317494 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef54f794-13f7-4b96-a57e-06ddbf9cb133 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:59.321101 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceecce5-3913-41fd-8eaf-c817e87c29ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:59.322031 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceecce5-3913-41fd-8eaf-c817e87c29ee service nova] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:59.322475 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceecce5-3913-41fd-8eaf-c817e87c29ee service nova] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:59.322919 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceecce5-3913-41fd-8eaf-c817e87c29ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:59.337550 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceecce5-3913-41fd-8eaf-c817e87c29ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:59.339576 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceecce5-3913-41fd-8eaf-c817e87c29ee service nova] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:59.373741 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 834/1684] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:30:59 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:30:59.455411 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 851/1685] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:30:59 2022] POST /identity/v3/projects => generated 363 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:30:59.459817 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceecce5-3913-41fd-8eaf-c817e87c29ee service nova] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:59.475165 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceecce5-3913-41fd-8eaf-c817e87c29ee service nova] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:59.650961 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c702e8-7298-4f16-984f-5e62f0b70111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:59.655744 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c702e8-7298-4f16-984f-5e62f0b70111 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:59.657130 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c702e8-7298-4f16-984f-5e62f0b70111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:59.657488 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c702e8-7298-4f16-984f-5e62f0b70111 admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:59.660622 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c702e8-7298-4f16-984f-5e62f0b70111 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:59.663230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c702e8-7298-4f16-984f-5e62f0b70111 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:59.799080 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 835/1686] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:30:59 2022] POST /identity/v3/users => generated 288 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:30:59.802395 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef54f794-13f7-4b96-a57e-06ddbf9cb133 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:59.834871 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef54f794-13f7-4b96-a57e-06ddbf9cb133 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:59.856995 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b857bf-fa7f-4325-9271-ae3c291e0700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:59.857991 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b857bf-fa7f-4325-9271-ae3c291e0700 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:59.864362 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b857bf-fa7f-4325-9271-ae3c291e0700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:59.865653 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b857bf-fa7f-4325-9271-ae3c291e0700 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:30:59.870169 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b857bf-fa7f-4325-9271-ae3c291e0700 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:30:59.874582 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b857bf-fa7f-4325-9271-ae3c291e0700 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:30:59.915286 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 852/1687] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:30:59.916666 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c702e8-7298-4f16-984f-5e62f0b70111 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:30:59.932685 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c702e8-7298-4f16-984f-5e62f0b70111 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:30:59.960015 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42251a-c000-4822-b662-3760ee2595f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:30:59.960760 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42251a-c000-4822-b662-3760ee2595f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:30:59.961049 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42251a-c000-4822-b662-3760ee2595f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:30:59.961331 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42251a-c000-4822-b662-3760ee2595f8 admin admin] PATH_INFO: `/v3/projects/6427bd29e4ed4a9f9398dabf662285fe/users/50cdac31a04a459cb350211d4731528e/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:00.007351 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:00.015911 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd42251a-c000-4822-b662-3760ee2595f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6427bd29e4ed4a9f9398dabf662285fe, user_id=50cdac31a04a459cb350211d4731528e, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:00.036316 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd42251a-c000-4822-b662-3760ee2595f8 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:00.149292 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 836/1688] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:30:59 2022] PUT /identity/v3/projects/6427bd29e4ed4a9f9398dabf662285fe/users/50cdac31a04a459cb350211d4731528e/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:00.153221 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b857bf-fa7f-4325-9271-ae3c291e0700 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:00.188710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b857bf-fa7f-4325-9271-ae3c291e0700 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:00.290785 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e39205-6a79-499a-918e-6de40fb9041f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:00.291876 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e39205-6a79-499a-918e-6de40fb9041f admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:00.292361 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e39205-6a79-499a-918e-6de40fb9041f admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:00.292784 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e39205-6a79-499a-918e-6de40fb9041f admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:00.293738 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e39205-6a79-499a-918e-6de40fb9041f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:00.295813 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e39205-6a79-499a-918e-6de40fb9041f admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:00.317354 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42251a-c000-4822-b662-3760ee2595f8 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:00.336139 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 853/1689] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:31:00 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:31:00.342865 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42251a-c000-4822-b662-3760ee2595f8 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:00.354344 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4e8ec-2af7-4c93-afb1-ecdff9b5be17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:00.355364 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4e8ec-2af7-4c93-afb1-ecdff9b5be17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:00.355856 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4e8ec-2af7-4c93-afb1-ecdff9b5be17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:00.356378 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4e8ec-2af7-4c93-afb1-ecdff9b5be17 admin admin] PATH_INFO: `/v3/projects/6427bd29e4ed4a9f9398dabf662285fe/users/50cdac31a04a459cb350211d4731528e/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:00.358857 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f4e8ec-2af7-4c93-afb1-ecdff9b5be17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6427bd29e4ed4a9f9398dabf662285fe, user_id=50cdac31a04a459cb350211d4731528e, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:00.362426 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f4e8ec-2af7-4c93-afb1-ecdff9b5be17 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:00.391022 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 837/1690] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:31:00 2022] PUT /identity/v3/projects/6427bd29e4ed4a9f9398dabf662285fe/users/50cdac31a04a459cb350211d4731528e/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:00.401829 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee31774-a702-417e-9d8e-e188bce38c13 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:00.402687 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee31774-a702-417e-9d8e-e188bce38c13 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:00.403339 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee31774-a702-417e-9d8e-e188bce38c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:00.482299 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-1ee31774-a702-417e-9d8e-e188bce38c13 None None] MFA Rules not processed for user `50cdac31a04a459cb350211d4731528e`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:00.487017 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-1ee31774-a702-417e-9d8e-e188bce38c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:00.523639 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573096d-bf00-4f33-9edc-4892ae6cd2a2 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:00.523855 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573096d-bf00-4f33-9edc-4892ae6cd2a2 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:00.524280 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573096d-bf00-4f33-9edc-4892ae6cd2a2 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:00.526721 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 838/1691] 10.222.0.40 () {54 vars in 912 bytes} [Fri Aug 5 15:31:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:31:00.533374 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e64cd39-108b-468f-b99e-c1af6a11a2a0 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:00.533776 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e64cd39-108b-468f-b99e-c1af6a11a2a0 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:00.534064 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e64cd39-108b-468f-b99e-c1af6a11a2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:00.616009 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:00.626798 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-5e64cd39-108b-468f-b99e-c1af6a11a2a0 None None] MFA Rules not processed for user `d13a1638b2ca497fb02c71615559d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:00.631076 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-5e64cd39-108b-468f-b99e-c1af6a11a2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:00.688650 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 854/1692] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:00 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:00.701151 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee31774-a702-417e-9d8e-e188bce38c13 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:00.706379 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-1ee31774-a702-417e-9d8e-e188bce38c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:00.760554 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 839/1693] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 5 15:31:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:00.773634 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e64cd39-108b-468f-b99e-c1af6a11a2a0 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:00.775347 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-5e64cd39-108b-468f-b99e-c1af6a11a2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:00.803309 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e64cd39-108b-468f-b99e-c1af6a11a2a0 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:00.812906 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911592ff-e6e5-47c6-a9bd-ec388ba63fd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:00.813616 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911592ff-e6e5-47c6-a9bd-ec388ba63fd8 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:00.813912 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911592ff-e6e5-47c6-a9bd-ec388ba63fd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:00.814196 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911592ff-e6e5-47c6-a9bd-ec388ba63fd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:00.826666 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911592ff-e6e5-47c6-a9bd-ec388ba63fd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:00.834288 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911592ff-e6e5-47c6-a9bd-ec388ba63fd8 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:00.838578 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 840/1694] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:31:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 15:31:00.843157 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee31774-a702-417e-9d8e-e188bce38c13 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:00.850384 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c837f7-ecbb-41fb-9ce4-36a20aec413c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:00.851206 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c837f7-ecbb-41fb-9ce4-36a20aec413c service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:00.851496 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c837f7-ecbb-41fb-9ce4-36a20aec413c service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:00.851779 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c837f7-ecbb-41fb-9ce4-36a20aec413c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:00.876795 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c837f7-ecbb-41fb-9ce4-36a20aec413c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:00.878052 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c837f7-ecbb-41fb-9ce4-36a20aec413c service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:00.902303 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 855/1695] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:31:00 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:31:01.465884 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204fa67d-4d4e-4c54-8bc9-0a3663ed7964 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:01.466423 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204fa67d-4d4e-4c54-8bc9-0a3663ed7964 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:01.466423 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204fa67d-4d4e-4c54-8bc9-0a3663ed7964 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:01.468148 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 841/1696] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 5 15:31:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:31:01.519607 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05533b31-1942-4d58-8f23-03929405b9f3 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:01.530749 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05533b31-1942-4d58-8f23-03929405b9f3 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:01.530749 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05533b31-1942-4d58-8f23-03929405b9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:01.564444 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-05533b31-1942-4d58-8f23-03929405b9f3 None None] MFA Rules not processed for user `107a7c18a2c6440b86e1b7c3e39cc724`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:01.566688 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-05533b31-1942-4d58-8f23-03929405b9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:01.801721 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4d2e5-18c0-4154-a832-2effc090a008 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:01.802012 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4d2e5-18c0-4154-a832-2effc090a008 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:01.802291 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4d2e5-18c0-4154-a832-2effc090a008 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:01.803642 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 842/1697] 10.222.0.40 () {54 vars in 961 bytes} [Fri Aug 5 15:31:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:31:01.807263 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 856/1698] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 5 15:31:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:01.810114 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aeb78a-1ce8-416a-82b7-50e5f28714ae None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:01.810575 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aeb78a-1ce8-416a-82b7-50e5f28714ae None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:01.811158 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aeb78a-1ce8-416a-82b7-50e5f28714ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:01.835625 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-87aeb78a-1ce8-416a-82b7-50e5f28714ae None None] MFA Rules not processed for user `107a7c18a2c6440b86e1b7c3e39cc724`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:01.837038 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-87aeb78a-1ce8-416a-82b7-50e5f28714ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:01.869316 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 843/1699] 10.222.0.40 () {58 vars in 1058 bytes} [Fri Aug 5 15:31:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:31:01.873347 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05533b31-1942-4d58-8f23-03929405b9f3 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:01.876021 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-05533b31-1942-4d58-8f23-03929405b9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:01.879129 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aeb78a-1ce8-416a-82b7-50e5f28714ae None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:01.890790 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aeb78a-1ce8-416a-82b7-50e5f28714ae None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:01.904459 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05533b31-1942-4d58-8f23-03929405b9f3 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:01.907950 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53331e2-6510-4ef9-914e-e263e9a3dc8a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '107a7c18a2c6440b86e1b7c3e39cc724', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:01.908634 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53331e2-6510-4ef9-914e-e263e9a3dc8a service placement] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:01.912550 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53331e2-6510-4ef9-914e-e263e9a3dc8a service placement] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:01.913181 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53331e2-6510-4ef9-914e-e263e9a3dc8a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:01.922347 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-a53331e2-6510-4ef9-914e-e263e9a3dc8a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:01.935714 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a091aab6-0af4-4a82-90fa-ec50a63c919e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '107a7c18a2c6440b86e1b7c3e39cc724', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:01.954084 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a091aab6-0af4-4a82-90fa-ec50a63c919e service placement] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:01.955534 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a091aab6-0af4-4a82-90fa-ec50a63c919e service placement] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:01.955829 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a091aab6-0af4-4a82-90fa-ec50a63c919e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:01.983064 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53331e2-6510-4ef9-914e-e263e9a3dc8a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:01.983549 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a091aab6-0af4-4a82-90fa-ec50a63c919e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:01.985308 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53331e2-6510-4ef9-914e-e263e9a3dc8a service placement] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:01.987313 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a091aab6-0af4-4a82-90fa-ec50a63c919e service placement] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:02.003976 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:02.014043 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 844/1700] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:31:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:31:02.015647 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31399767-3cc8-4323-abd1-9301b46633cf None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:02.016244 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31399767-3cc8-4323-abd1-9301b46633cf None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:02.016750 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31399767-3cc8-4323-abd1-9301b46633cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:02.017445 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 857/1701] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:31:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:31:02.051469 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-31399767-3cc8-4323-abd1-9301b46633cf None None] MFA Rules not processed for user `cb055110c77543c1861dfd8c155d7582`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:02.053073 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-31399767-3cc8-4323-abd1-9301b46633cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:02.172728 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:02.329334 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 845/1702] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:02 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:02.611384 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a091aab6-0af4-4a82-90fa-ec50a63c919e service placement] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:02.856974 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a091aab6-0af4-4a82-90fa-ec50a63c919e service placement] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:02.870705 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f34bf-c523-41fa-b61a-aeaa0cb68800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:02.871352 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f34bf-c523-41fa-b61a-aeaa0cb68800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:02.872611 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f34bf-c523-41fa-b61a-aeaa0cb68800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:02.876268 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f34bf-c523-41fa-b61a-aeaa0cb68800 admin admin] PATH_INFO: `/v3/users/3e51c27465d34d14b022425b95ad8734` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:02.876268 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782f34bf-c523-41fa-b61a-aeaa0cb68800 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e51c27465d34d14b022425b95ad8734)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:02.879158 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782f34bf-c523-41fa-b61a-aeaa0cb68800 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:02.918631 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31399767-3cc8-4323-abd1-9301b46633cf None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:02.920164 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-31399767-3cc8-4323-abd1-9301b46633cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:02.973827 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31399767-3cc8-4323-abd1-9301b46633cf None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:02.998534 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e198ec-98ca-4f23-9cbf-d6a137d50728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:02.999269 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e198ec-98ca-4f23-9cbf-d6a137d50728 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:02.999511 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e198ec-98ca-4f23-9cbf-d6a137d50728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:02.999774 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e198ec-98ca-4f23-9cbf-d6a137d50728 admin admin] PATH_INFO: `/v3/users/3e26066d56b74a3ca5fed6b180f5c689` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:03.018534 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e198ec-98ca-4f23-9cbf-d6a137d50728 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e26066d56b74a3ca5fed6b180f5c689)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:03.021187 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e198ec-98ca-4f23-9cbf-d6a137d50728 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:03.255338 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-782f34bf-c523-41fa-b61a-aeaa0cb68800 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e51c27465d34d14b022425b95ad8734'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:03.270846 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-782f34bf-c523-41fa-b61a-aeaa0cb68800 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e51c27465d34d14b022425b95ad8734'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:03.330065 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-90e198ec-98ca-4f23-9cbf-d6a137d50728 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e26066d56b74a3ca5fed6b180f5c689'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:03.343744 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-782f34bf-c523-41fa-b61a-aeaa0cb68800 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e51c27465d34d14b022425b95ad8734'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:03.365262 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-90e198ec-98ca-4f23-9cbf-d6a137d50728 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e26066d56b74a3ca5fed6b180f5c689'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:03.368510 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-90e198ec-98ca-4f23-9cbf-d6a137d50728 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e26066d56b74a3ca5fed6b180f5c689'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:03.421504 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-90e198ec-98ca-4f23-9cbf-d6a137d50728 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e26066d56b74a3ca5fed6b180f5c689'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:03.422362 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-90e198ec-98ca-4f23-9cbf-d6a137d50728 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e26066d56b74a3ca5fed6b180f5c689'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:03.435756 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-782f34bf-c523-41fa-b61a-aeaa0cb68800 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e51c27465d34d14b022425b95ad8734'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:03.447920 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-782f34bf-c523-41fa-b61a-aeaa0cb68800 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e51c27465d34d14b022425b95ad8734'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:03.468037 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 858/1703] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:31:02 2022] DELETE /identity/v3/users/3e51c27465d34d14b022425b95ad8734 => generated 0 bytes in 862 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:31:03.474157 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71abc18-a3db-4eb4-8a50-bd571aac75cc None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:03.478078 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71abc18-a3db-4eb4-8a50-bd571aac75cc None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:03.478078 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71abc18-a3db-4eb4-8a50-bd571aac75cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:03.524393 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-b71abc18-a3db-4eb4-8a50-bd571aac75cc None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:03.541148 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-b71abc18-a3db-4eb4-8a50-bd571aac75cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:03.713248 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 846/1704] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:31:02 2022] DELETE /identity/v3/users/3e26066d56b74a3ca5fed6b180f5c689 => generated 0 bytes in 681 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:03.716734 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 859/1705] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:31:03.717915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4362c-4d7e-4ebb-a0f6-961fff06e0df None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:03.718545 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4362c-4d7e-4ebb-a0f6-961fff06e0df None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:03.718974 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4362c-4d7e-4ebb-a0f6-961fff06e0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:03.746937 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71abc18-a3db-4eb4-8a50-bd571aac75cc None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:03.747812 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-3bd4362c-4d7e-4ebb-a0f6-961fff06e0df None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:03.748474 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-b71abc18-a3db-4eb4-8a50-bd571aac75cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:03.749840 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-3bd4362c-4d7e-4ebb-a0f6-961fff06e0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:03.788630 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:03.859711 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71abc18-a3db-4eb4-8a50-bd571aac75cc None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:03.946600 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 847/1706] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:04.066653 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a1e393-0761-4aa1-abce-693fe41226d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:04.068699 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1e393-0761-4aa1-abce-693fe41226d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:04.071896 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1e393-0761-4aa1-abce-693fe41226d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:04.072587 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1e393-0761-4aa1-abce-693fe41226d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:04.088349 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a1e393-0761-4aa1-abce-693fe41226d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:04.091492 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a1e393-0761-4aa1-abce-693fe41226d0 service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:04.095029 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4362c-4d7e-4ebb-a0f6-961fff06e0df None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:04.107449 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4362c-4d7e-4ebb-a0f6-961fff06e0df None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:04.117764 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 860/1707] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:31:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:31:04.121148 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254ffed3-d48c-4d0f-93f8-9d2b4ce60537 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:04.124361 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ffed3-d48c-4d0f-93f8-9d2b4ce60537 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:04.124361 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ffed3-d48c-4d0f-93f8-9d2b4ce60537 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:04.124361 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ffed3-d48c-4d0f-93f8-9d2b4ce60537 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:04.143088 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254ffed3-d48c-4d0f-93f8-9d2b4ce60537 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:04.144678 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254ffed3-d48c-4d0f-93f8-9d2b4ce60537 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:04.187954 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 848/1708] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:31:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:31:04.940135 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8ca59-5dee-46a1-8113-222c6581626c None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:04.941282 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8ca59-5dee-46a1-8113-222c6581626c None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:04.941377 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8ca59-5dee-46a1-8113-222c6581626c None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:04.942695 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 861/1709] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 5 15:31:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:31:04.995949 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42350e19-630f-4893-b59d-2657ac34a400 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:04.996537 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42350e19-630f-4893-b59d-2657ac34a400 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:04.996537 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42350e19-630f-4893-b59d-2657ac34a400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:05.035693 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-42350e19-630f-4893-b59d-2657ac34a400 None None] MFA Rules not processed for user `107a7c18a2c6440b86e1b7c3e39cc724`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:05.037190 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-42350e19-630f-4893-b59d-2657ac34a400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:05.110757 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:05.188796 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 849/1710] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 5 15:31:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:05.208532 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba8ca59-5dee-46a1-8113-222c6581626c None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:05.209693 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-9ba8ca59-5dee-46a1-8113-222c6581626c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:05.232697 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba8ca59-5dee-46a1-8113-222c6581626c None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:05.239646 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad7a9fb-b446-4353-b5df-d56d3e2defb5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '107a7c18a2c6440b86e1b7c3e39cc724', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:05.240644 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7a9fb-b446-4353-b5df-d56d3e2defb5 service placement] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:05.241148 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7a9fb-b446-4353-b5df-d56d3e2defb5 service placement] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:05.241630 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7a9fb-b446-4353-b5df-d56d3e2defb5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:05.259998 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad7a9fb-b446-4353-b5df-d56d3e2defb5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:05.262533 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad7a9fb-b446-4353-b5df-d56d3e2defb5 service placement] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:05.312307 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 862/1711] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:31:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:31:05.373107 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42350e19-630f-4893-b59d-2657ac34a400 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:05.374629 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-42350e19-630f-4893-b59d-2657ac34a400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:05.401192 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42350e19-630f-4893-b59d-2657ac34a400 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:05.429021 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e2c3b-843b-40ed-b89b-0e2787635d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:05.429714 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e2c3b-843b-40ed-b89b-0e2787635d93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:05.430032 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e2c3b-843b-40ed-b89b-0e2787635d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:05.430274 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e2c3b-843b-40ed-b89b-0e2787635d93 admin admin] PATH_INFO: `/v3/users/ec7fc07d65854cb18789d6cfe8e7d710` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:05.432024 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12e2c3b-843b-40ed-b89b-0e2787635d93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec7fc07d65854cb18789d6cfe8e7d710)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:05.435986 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12e2c3b-843b-40ed-b89b-0e2787635d93 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:05.717323 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4294a8-3022-4680-8cea-ded620ef5d82 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:05.717626 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4294a8-3022-4680-8cea-ded620ef5d82 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:05.718000 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4294a8-3022-4680-8cea-ded620ef5d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:05.740262 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-e12e2c3b-843b-40ed-b89b-0e2787635d93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec7fc07d65854cb18789d6cfe8e7d710'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:05.753508 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-e12e2c3b-843b-40ed-b89b-0e2787635d93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec7fc07d65854cb18789d6cfe8e7d710'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:05.755071 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-e12e2c3b-843b-40ed-b89b-0e2787635d93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec7fc07d65854cb18789d6cfe8e7d710'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:05.782623 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-e12e2c3b-843b-40ed-b89b-0e2787635d93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec7fc07d65854cb18789d6cfe8e7d710'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:05.783508 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-e12e2c3b-843b-40ed-b89b-0e2787635d93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec7fc07d65854cb18789d6cfe8e7d710'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:05.787988 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-ee4294a8-3022-4680-8cea-ded620ef5d82 None None] MFA Rules not processed for user `b0882cb9bf4744feae298fb8f10cebeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:05.789673 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-ee4294a8-3022-4680-8cea-ded620ef5d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:05.903904 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 850/1712] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:31:05 2022] DELETE /identity/v3/users/ec7fc07d65854cb18789d6cfe8e7d710 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:05.907875 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e2c3b-843b-40ed-b89b-0e2787635d93 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:05.909683 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-e12e2c3b-843b-40ed-b89b-0e2787635d93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:06.027567 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 863/1713] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:06.037914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941563f-dcc2-4d96-8372-fb2ef1ea8c95 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:06.038372 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941563f-dcc2-4d96-8372-fb2ef1ea8c95 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:06.038801 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941563f-dcc2-4d96-8372-fb2ef1ea8c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:06.113866 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-0941563f-dcc2-4d96-8372-fb2ef1ea8c95 None None] MFA Rules not processed for user `b0882cb9bf4744feae298fb8f10cebeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:06.113866 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-0941563f-dcc2-4d96-8372-fb2ef1ea8c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:06.126110 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e2c3b-843b-40ed-b89b-0e2787635d93 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:06.149568 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc310ed0-32b6-493c-9523-a89c106ea110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:06.150504 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc310ed0-32b6-493c-9523-a89c106ea110 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:06.150941 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc310ed0-32b6-493c-9523-a89c106ea110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:06.151369 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc310ed0-32b6-493c-9523-a89c106ea110 admin admin] PATH_INFO: `/v3/projects/383dd2344dc4403aa7f4f0953435ddc3` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:06.152829 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc310ed0-32b6-493c-9523-a89c106ea110 admin admin] RBAC: Authorizing `identity:delete_project(project_id=383dd2344dc4403aa7f4f0953435ddc3)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:06.154015 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: WARNING py.warnings [None req-dc310ed0-32b6-493c-9523-a89c106ea110 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 15:31:06.154015 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: warnings.warn(msg) Aug 05 15:31:06.154015 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: Aug 05 15:31:06.155660 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc310ed0-32b6-493c-9523-a89c106ea110 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:06.236335 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:06.264730 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 864/1714] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:06.278544 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0941563f-dcc2-4d96-8372-fb2ef1ea8c95 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:06.282720 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-0941563f-dcc2-4d96-8372-fb2ef1ea8c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:06.370823 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:06.390578 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-dc310ed0-32b6-493c-9523-a89c106ea110 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '383dd2344dc4403aa7f4f0953435ddc3', 'request_id': 'req-dc310ed0-32b6-493c-9523-a89c106ea110'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:06.414723 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-dc310ed0-32b6-493c-9523-a89c106ea110 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '383dd2344dc4403aa7f4f0953435ddc3', 'request_id': 'req-dc310ed0-32b6-493c-9523-a89c106ea110'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:06.418532 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0941563f-dcc2-4d96-8372-fb2ef1ea8c95 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:06.504307 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-dc310ed0-32b6-493c-9523-a89c106ea110 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '383dd2344dc4403aa7f4f0953435ddc3', 'request_id': 'req-dc310ed0-32b6-493c-9523-a89c106ea110'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:06.525766 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-dc310ed0-32b6-493c-9523-a89c106ea110 admin admin] The token cache is being invalidate because project 383dd2344dc4403aa7f4f0953435ddc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:31:06.526809 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-dc310ed0-32b6-493c-9523-a89c106ea110 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:06.527501 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-dc310ed0-32b6-493c-9523-a89c106ea110 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:06.531576 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 851/1715] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:31:05 2022] DELETE /identity/v3/projects/383dd2344dc4403aa7f4f0953435ddc3 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:06.532453 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc310ed0-32b6-493c-9523-a89c106ea110 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:06.533929 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-dc310ed0-32b6-493c-9523-a89c106ea110 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:06.603151 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fedc0f6-d1a1-4e07-8c54-ede1a98be1bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:06.604445 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedc0f6-d1a1-4e07-8c54-ede1a98be1bf service nova] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:06.604974 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedc0f6-d1a1-4e07-8c54-ede1a98be1bf service nova] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:06.605479 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedc0f6-d1a1-4e07-8c54-ede1a98be1bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:06.618150 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-6fedc0f6-d1a1-4e07-8c54-ede1a98be1bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:06.684773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc310ed0-32b6-493c-9523-a89c106ea110 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:06.691304 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ceca0-dbcb-48d3-839c-1dcecf51441a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:06.692143 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ceca0-dbcb-48d3-839c-1dcecf51441a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:06.692254 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ceca0-dbcb-48d3-839c-1dcecf51441a admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:06.692465 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ceca0-dbcb-48d3-839c-1dcecf51441a admin admin] PATH_INFO: `/v3/projects/75f1e9851c4945a599dd0843f17c2530` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:06.693833 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2ceca0-dbcb-48d3-839c-1dcecf51441a admin admin] RBAC: Authorizing `identity:delete_project(project_id=75f1e9851c4945a599dd0843f17c2530)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:06.695850 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2ceca0-dbcb-48d3-839c-1dcecf51441a admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:06.748734 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fedc0f6-d1a1-4e07-8c54-ede1a98be1bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:06.751515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fedc0f6-d1a1-4e07-8c54-ede1a98be1bf service nova] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:06.769040 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 865/1716] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:31:06 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:31:06.769304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdedb0a-3dc8-4eca-80ca-42e81ad4a8b9 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:06.769960 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdedb0a-3dc8-4eca-80ca-42e81ad4a8b9 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:06.770735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdedb0a-3dc8-4eca-80ca-42e81ad4a8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:06.818629 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-1cdedb0a-3dc8-4eca-80ca-42e81ad4a8b9 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:06.822091 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-1cdedb0a-3dc8-4eca-80ca-42e81ad4a8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:06.839972 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 866/1717] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:06.841271 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdedb0a-3dc8-4eca-80ca-42e81ad4a8b9 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:06.842988 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-1cdedb0a-3dc8-4eca-80ca-42e81ad4a8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:06.869843 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-4d2ceca0-dbcb-48d3-839c-1dcecf51441a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75f1e9851c4945a599dd0843f17c2530', 'request_id': 'req-4d2ceca0-dbcb-48d3-839c-1dcecf51441a'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:06.891490 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-4d2ceca0-dbcb-48d3-839c-1dcecf51441a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75f1e9851c4945a599dd0843f17c2530', 'request_id': 'req-4d2ceca0-dbcb-48d3-839c-1dcecf51441a'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:06.895203 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdedb0a-3dc8-4eca-80ca-42e81ad4a8b9 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:06.969474 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:06.984829 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-4d2ceca0-dbcb-48d3-839c-1dcecf51441a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75f1e9851c4945a599dd0843f17c2530', 'request_id': 'req-4d2ceca0-dbcb-48d3-839c-1dcecf51441a'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:07.006527 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-4d2ceca0-dbcb-48d3-839c-1dcecf51441a admin admin] The token cache is being invalidate because project 75f1e9851c4945a599dd0843f17c2530 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:31:07.007061 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-4d2ceca0-dbcb-48d3-839c-1dcecf51441a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:07.008028 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-4d2ceca0-dbcb-48d3-839c-1dcecf51441a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:07.040446 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f01a284-b5a9-4012-883e-ccaef20a038e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:07.041363 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01a284-b5a9-4012-883e-ccaef20a038e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:07.041715 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01a284-b5a9-4012-883e-ccaef20a038e admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:07.042116 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01a284-b5a9-4012-883e-ccaef20a038e admin admin] PATH_INFO: `/v3/projects/3c58b93a5db6487cb0b3061ab3f0aa1c` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:07.043574 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f01a284-b5a9-4012-883e-ccaef20a038e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c58b93a5db6487cb0b3061ab3f0aa1c)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:07.045265 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 852/1718] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:31:06 2022] DELETE /identity/v3/projects/75f1e9851c4945a599dd0843f17c2530 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:07.048933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e547d8-965c-4bdd-88b6-2b5ecfbecb25 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:07.049109 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f01a284-b5a9-4012-883e-ccaef20a038e admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:07.049644 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e547d8-965c-4bdd-88b6-2b5ecfbecb25 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:07.050646 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e547d8-965c-4bdd-88b6-2b5ecfbecb25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:07.084469 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-76e547d8-965c-4bdd-88b6-2b5ecfbecb25 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:07.086262 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-76e547d8-965c-4bdd-88b6-2b5ecfbecb25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:07.097821 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 853/1719] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:07.098997 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e547d8-965c-4bdd-88b6-2b5ecfbecb25 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:07.102179 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-76e547d8-965c-4bdd-88b6-2b5ecfbecb25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:07.119111 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e547d8-965c-4bdd-88b6-2b5ecfbecb25 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:07.124976 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0affda-1ca7-419b-924a-0fe924e92a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:07.126835 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0affda-1ca7-419b-924a-0fe924e92a2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:07.127931 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0affda-1ca7-419b-924a-0fe924e92a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:07.128623 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0affda-1ca7-419b-924a-0fe924e92a2b admin admin] PATH_INFO: `/v3/users/8b2513d0c46c4a2b9585ce2ad7d705de` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:07.130321 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0affda-1ca7-419b-924a-0fe924e92a2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b2513d0c46c4a2b9585ce2ad7d705de)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:07.132931 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0affda-1ca7-419b-924a-0fe924e92a2b admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:07.318786 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-6f01a284-b5a9-4012-883e-ccaef20a038e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c58b93a5db6487cb0b3061ab3f0aa1c', 'request_id': 'req-6f01a284-b5a9-4012-883e-ccaef20a038e'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:07.341825 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-6f01a284-b5a9-4012-883e-ccaef20a038e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c58b93a5db6487cb0b3061ab3f0aa1c', 'request_id': 'req-6f01a284-b5a9-4012-883e-ccaef20a038e'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:07.353382 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-6f01a284-b5a9-4012-883e-ccaef20a038e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c58b93a5db6487cb0b3061ab3f0aa1c', 'request_id': 'req-6f01a284-b5a9-4012-883e-ccaef20a038e'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:07.422170 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-fe0affda-1ca7-419b-924a-0fe924e92a2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b2513d0c46c4a2b9585ce2ad7d705de'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:07.445337 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:07.459693 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-6f01a284-b5a9-4012-883e-ccaef20a038e admin admin] The token cache is being invalidate because project 3c58b93a5db6487cb0b3061ab3f0aa1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:31:07.462444 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-6f01a284-b5a9-4012-883e-ccaef20a038e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:07.462568 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-6f01a284-b5a9-4012-883e-ccaef20a038e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:07.464970 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 867/1720] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:31:06 2022] DELETE /identity/v3/projects/3c58b93a5db6487cb0b3061ab3f0aa1c => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:07.465938 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f01a284-b5a9-4012-883e-ccaef20a038e admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:07.471686 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-fe0affda-1ca7-419b-924a-0fe924e92a2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b2513d0c46c4a2b9585ce2ad7d705de'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:07.472144 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-6f01a284-b5a9-4012-883e-ccaef20a038e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:07.474176 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-fe0affda-1ca7-419b-924a-0fe924e92a2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b2513d0c46c4a2b9585ce2ad7d705de'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:07.543608 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-fe0affda-1ca7-419b-924a-0fe924e92a2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b2513d0c46c4a2b9585ce2ad7d705de'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:07.559561 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-fe0affda-1ca7-419b-924a-0fe924e92a2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b2513d0c46c4a2b9585ce2ad7d705de'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:07.622164 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f01a284-b5a9-4012-883e-ccaef20a038e admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:07.633103 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f96deb2-3315-4ab2-9d3c-bf8417bf3644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:07.642453 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 854/1721] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:31:07 2022] DELETE /identity/v3/users/8b2513d0c46c4a2b9585ce2ad7d705de => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:07.653936 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0affda-1ca7-419b-924a-0fe924e92a2b admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:07.657758 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-fe0affda-1ca7-419b-924a-0fe924e92a2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:07.661721 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f96deb2-3315-4ab2-9d3c-bf8417bf3644 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:07.669431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f96deb2-3315-4ab2-9d3c-bf8417bf3644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:07.669431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f96deb2-3315-4ab2-9d3c-bf8417bf3644 admin admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:07.669431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f96deb2-3315-4ab2-9d3c-bf8417bf3644 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:31:07.669431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f96deb2-3315-4ab2-9d3c-bf8417bf3644 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:07.676400 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f96deb2-3315-4ab2-9d3c-bf8417bf3644 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:07.716887 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 868/1722] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:31:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:31:07.716887 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd1cf1-eb41-4197-affa-d8f4314587c1 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:07.716887 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd1cf1-eb41-4197-affa-d8f4314587c1 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:07.716887 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd1cf1-eb41-4197-affa-d8f4314587c1 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:07.716887 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 869/1723] 10.222.0.40 () {54 vars in 912 bytes} [Fri Aug 5 15:31:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:31:07.716887 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd1cf1-eb41-4197-affa-d8f4314587c1 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:07.728528 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-3cbd1cf1-eb41-4197-affa-d8f4314587c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:07.805568 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:07.822436 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd1cf1-eb41-4197-affa-d8f4314587c1 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:07.830093 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4da3d5c-0635-4f5c-bef9-eb59d4169cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:07.830093 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da3d5c-0635-4f5c-bef9-eb59d4169cc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:07.830093 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da3d5c-0635-4f5c-bef9-eb59d4169cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:07.830453 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da3d5c-0635-4f5c-bef9-eb59d4169cc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:07.831357 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4da3d5c-0635-4f5c-bef9-eb59d4169cc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:07.834293 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4da3d5c-0635-4f5c-bef9-eb59d4169cc8 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:07.895228 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0affda-1ca7-419b-924a-0fe924e92a2b admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:07.907936 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a22bb29-567f-4dbd-aaec-0cfde22d9f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:07.908670 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22bb29-567f-4dbd-aaec-0cfde22d9f7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:07.909002 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22bb29-567f-4dbd-aaec-0cfde22d9f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:07.909295 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22bb29-567f-4dbd-aaec-0cfde22d9f7b admin admin] PATH_INFO: `/v3/users/50cdac31a04a459cb350211d4731528e` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:07.910916 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a22bb29-567f-4dbd-aaec-0cfde22d9f7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=50cdac31a04a459cb350211d4731528e)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:07.914723 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a22bb29-567f-4dbd-aaec-0cfde22d9f7b admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:08.110480 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 870/1724] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:31:07 2022] POST /identity/v3/projects => generated 403 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:31:08.113984 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed46d3-835e-4115-8abe-9332f323ef5e None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:08.114488 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed46d3-835e-4115-8abe-9332f323ef5e None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:08.114958 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed46d3-835e-4115-8abe-9332f323ef5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:08.158108 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-4a22bb29-567f-4dbd-aaec-0cfde22d9f7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50cdac31a04a459cb350211d4731528e'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:08.166165 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-4a22bb29-567f-4dbd-aaec-0cfde22d9f7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50cdac31a04a459cb350211d4731528e'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:08.167465 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-4a22bb29-567f-4dbd-aaec-0cfde22d9f7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50cdac31a04a459cb350211d4731528e'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:08.173013 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-c2ed46d3-835e-4115-8abe-9332f323ef5e None None] MFA Rules not processed for user `d13a1638b2ca497fb02c71615559d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:08.173680 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-c2ed46d3-835e-4115-8abe-9332f323ef5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:08.184989 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-4a22bb29-567f-4dbd-aaec-0cfde22d9f7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50cdac31a04a459cb350211d4731528e'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:08.186676 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-4a22bb29-567f-4dbd-aaec-0cfde22d9f7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50cdac31a04a459cb350211d4731528e'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:08.266031 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 855/1725] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:31:07 2022] DELETE /identity/v3/users/50cdac31a04a459cb350211d4731528e => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:08.267984 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2591c43-f5bd-4423-b253-e3899fc90d1e None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:08.268496 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2591c43-f5bd-4423-b253-e3899fc90d1e None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:08.268909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2591c43-f5bd-4423-b253-e3899fc90d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:08.276541 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 871/1726] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 5 15:31:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:08.277610 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ed46d3-835e-4115-8abe-9332f323ef5e None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:08.279508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-c2ed46d3-835e-4115-8abe-9332f323ef5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:08.431150 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-c2591c43-f5bd-4423-b253-e3899fc90d1e None None] MFA Rules not processed for user `04e80222936d4427909414a78e1dc489`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:08.432358 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-c2591c43-f5bd-4423-b253-e3899fc90d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:08.444570 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ed46d3-835e-4115-8abe-9332f323ef5e None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:08.549960 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:08.602786 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeea186d-79e8-4499-8408-404a3986368c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:08.602786 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeea186d-79e8-4499-8408-404a3986368c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:08.602786 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeea186d-79e8-4499-8408-404a3986368c admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:08.602786 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeea186d-79e8-4499-8408-404a3986368c admin admin] PATH_INFO: `/v3/projects/05b57b1a5cde4522a3517babee3cf463` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:08.605330 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeea186d-79e8-4499-8408-404a3986368c admin admin] RBAC: Authorizing `identity:delete_project(project_id=05b57b1a5cde4522a3517babee3cf463)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:08.612397 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeea186d-79e8-4499-8408-404a3986368c admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:08.723316 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 856/1727] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 457 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:08.731223 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2591c43-f5bd-4423-b253-e3899fc90d1e None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:08.735331 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-c2591c43-f5bd-4423-b253-e3899fc90d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:08.768132 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2591c43-f5bd-4423-b253-e3899fc90d1e None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:08.787087 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ba5615-85d4-435a-84f6-520f8aeec845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:08.788463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba5615-85d4-435a-84f6-520f8aeec845 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:08.788463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba5615-85d4-435a-84f6-520f8aeec845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:08.788463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba5615-85d4-435a-84f6-520f8aeec845 admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:08.789168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ba5615-85d4-435a-84f6-520f8aeec845 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:08.791162 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ba5615-85d4-435a-84f6-520f8aeec845 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:08.829097 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-aeea186d-79e8-4499-8408-404a3986368c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05b57b1a5cde4522a3517babee3cf463', 'request_id': 'req-aeea186d-79e8-4499-8408-404a3986368c'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:08.843218 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-aeea186d-79e8-4499-8408-404a3986368c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05b57b1a5cde4522a3517babee3cf463', 'request_id': 'req-aeea186d-79e8-4499-8408-404a3986368c'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:08.856138 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-aeea186d-79e8-4499-8408-404a3986368c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05b57b1a5cde4522a3517babee3cf463', 'request_id': 'req-aeea186d-79e8-4499-8408-404a3986368c'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:08.950929 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 857/1728] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:31:08 2022] POST /identity/v3/users => generated 302 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:31:08.950929 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ba5615-85d4-435a-84f6-520f8aeec845 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:08.952092 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-94ba5615-85d4-435a-84f6-520f8aeec845 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:08.965935 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-aeea186d-79e8-4499-8408-404a3986368c admin admin] The token cache is being invalidate because project 05b57b1a5cde4522a3517babee3cf463 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:31:08.973023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-aeea186d-79e8-4499-8408-404a3986368c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:08.973023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-aeea186d-79e8-4499-8408-404a3986368c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:08.973533 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 872/1729] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:31:08 2022] DELETE /identity/v3/projects/05b57b1a5cde4522a3517babee3cf463 => generated 0 bytes in 697 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:08.974521 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeea186d-79e8-4499-8408-404a3986368c admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:08.979349 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-aeea186d-79e8-4499-8408-404a3986368c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:09.110740 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ba5615-85d4-435a-84f6-520f8aeec845 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:09.149373 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeea186d-79e8-4499-8408-404a3986368c admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:09.180167 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d4aafe-9428-4896-90bf-5bebfcb3602a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:09.180902 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4aafe-9428-4896-90bf-5bebfcb3602a service nova] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:09.181260 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4aafe-9428-4896-90bf-5bebfcb3602a service nova] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:09.181706 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4aafe-9428-4896-90bf-5bebfcb3602a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:09.186767 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-22d4aafe-9428-4896-90bf-5bebfcb3602a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:09.226560 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7798f13a-6a65-4951-af1d-7069d6a8dd92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:09.227209 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798f13a-6a65-4951-af1d-7069d6a8dd92 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:09.227583 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798f13a-6a65-4951-af1d-7069d6a8dd92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:09.228025 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798f13a-6a65-4951-af1d-7069d6a8dd92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:09.233135 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-7798f13a-6a65-4951-af1d-7069d6a8dd92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:09.298128 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7798f13a-6a65-4951-af1d-7069d6a8dd92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:09.302553 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7798f13a-6a65-4951-af1d-7069d6a8dd92 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:09.324317 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d4aafe-9428-4896-90bf-5bebfcb3602a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:09.325754 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 858/1730] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:31:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:31:09.326599 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d4aafe-9428-4896-90bf-5bebfcb3602a service nova] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:09.326823 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7798f13a-6a65-4951-af1d-7069d6a8dd92 service neutron] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:09.328613 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-7798f13a-6a65-4951-af1d-7069d6a8dd92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:09.347210 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 873/1731] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:31:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:31:09.348659 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199d51a-a15b-42b0-9703-5f533e8fec41 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:09.348960 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199d51a-a15b-42b0-9703-5f533e8fec41 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:09.349245 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199d51a-a15b-42b0-9703-5f533e8fec41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:09.385197 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-d199d51a-a15b-42b0-9703-5f533e8fec41 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:09.386782 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-d199d51a-a15b-42b0-9703-5f533e8fec41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:09.429388 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:09.556480 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 874/1732] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:09.561006 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727d86a-cb24-4cfc-a9f8-32459515fe02 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:09.561305 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727d86a-cb24-4cfc-a9f8-32459515fe02 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:09.561591 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727d86a-cb24-4cfc-a9f8-32459515fe02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:09.623890 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7798f13a-6a65-4951-af1d-7069d6a8dd92 service neutron] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:09.631621 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b2c461-d7b1-42c5-af96-9f828e9eebff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:09.632334 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2c461-d7b1-42c5-af96-9f828e9eebff admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:09.632628 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2c461-d7b1-42c5-af96-9f828e9eebff admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:09.632912 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2c461-d7b1-42c5-af96-9f828e9eebff admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:09.633681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b2c461-d7b1-42c5-af96-9f828e9eebff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:09.639137 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b2c461-d7b1-42c5-af96-9f828e9eebff admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:09.667093 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 859/1733] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:31:09.668467 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437afe5-c8c3-4187-a64f-af42076571e0 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:09.668763 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437afe5-c8c3-4187-a64f-af42076571e0 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:09.669056 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437afe5-c8c3-4187-a64f-af42076571e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:09.669857 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-a727d86a-cb24-4cfc-a9f8-32459515fe02 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:09.671199 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-a727d86a-cb24-4cfc-a9f8-32459515fe02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:09.719068 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 875/1734] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:09.723454 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a727d86a-cb24-4cfc-a9f8-32459515fe02 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:09.753859 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a727d86a-cb24-4cfc-a9f8-32459515fe02 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:09.776587 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea4c47-a592-4bdb-ac76-24c36de20bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:09.777245 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea4c47-a592-4bdb-ac76-24c36de20bfd admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:09.777534 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea4c47-a592-4bdb-ac76-24c36de20bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:09.777825 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea4c47-a592-4bdb-ac76-24c36de20bfd admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:09.778695 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feea4c47-a592-4bdb-ac76-24c36de20bfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:09.780832 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feea4c47-a592-4bdb-ac76-24c36de20bfd admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:09.835175 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 876/1735] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:31:09.849564 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea4c47-a592-4bdb-ac76-24c36de20bfd admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:09.863110 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea4c47-a592-4bdb-ac76-24c36de20bfd admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:09.874937 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22505769-c2a0-452c-be3b-84671d7035fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:09.875573 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22505769-c2a0-452c-be3b-84671d7035fa admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:09.875864 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22505769-c2a0-452c-be3b-84671d7035fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:09.876174 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22505769-c2a0-452c-be3b-84671d7035fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:09.876896 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22505769-c2a0-452c-be3b-84671d7035fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:31:09.877584 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22505769-c2a0-452c-be3b-84671d7035fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:09.881627 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22505769-c2a0-452c-be3b-84671d7035fa admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:09.902769 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-7437afe5-c8c3-4187-a64f-af42076571e0 None None] MFA Rules not processed for user `1654fb6a81e14d6b99ab495bb6d029d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:09.905344 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-7437afe5-c8c3-4187-a64f-af42076571e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:09.915727 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 877/1736] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:31:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:31:09.916611 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22505769-c2a0-452c-be3b-84671d7035fa admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:09.932626 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22505769-c2a0-452c-be3b-84671d7035fa admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:09.939110 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eb9a5a-494f-4e5f-943c-951f31c40462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:09.941242 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb9a5a-494f-4e5f-943c-951f31c40462 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:09.941242 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb9a5a-494f-4e5f-943c-951f31c40462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:09.941242 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb9a5a-494f-4e5f-943c-951f31c40462 admin admin] PATH_INFO: `/v3/projects/3fb76d2caec94893b1ecb7a9b0e96986/users/ef1e3ac4feb245e3bb710d7dd43e5fb7/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:09.988120 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eb9a5a-494f-4e5f-943c-951f31c40462 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fb76d2caec94893b1ecb7a9b0e96986, user_id=ef1e3ac4feb245e3bb710d7dd43e5fb7, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:09.994443 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eb9a5a-494f-4e5f-943c-951f31c40462 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:10.106979 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 860/1737] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 5 15:31:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 439 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:10.109455 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7437afe5-c8c3-4187-a64f-af42076571e0 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:10.125083 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 878/1738] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:31:09 2022] PUT /identity/v3/projects/3fb76d2caec94893b1ecb7a9b0e96986/users/ef1e3ac4feb245e3bb710d7dd43e5fb7/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:10.137165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bd6d7-27ec-4a31-8932-48bf151e8ed6 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:10.137459 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bd6d7-27ec-4a31-8932-48bf151e8ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:10.138393 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7437afe5-c8c3-4187-a64f-af42076571e0 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:10.140305 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bd6d7-27ec-4a31-8932-48bf151e8ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:10.231555 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-bb8bd6d7-27ec-4a31-8932-48bf151e8ed6 None None] MFA Rules not processed for user `ef1e3ac4feb245e3bb710d7dd43e5fb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:10.234766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-bb8bd6d7-27ec-4a31-8932-48bf151e8ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:10.327939 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:10.336913 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c038c-046f-41b0-9b5d-22140f380634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:10.337560 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c038c-046f-41b0-9b5d-22140f380634 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:10.338017 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c038c-046f-41b0-9b5d-22140f380634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:10.338445 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c038c-046f-41b0-9b5d-22140f380634 admin admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:10.340284 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554c038c-046f-41b0-9b5d-22140f380634 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:10.347211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554c038c-046f-41b0-9b5d-22140f380634 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:10.366700 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 879/1739] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:10 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:10.368700 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1377db08-85eb-4023-bd63-8a7ebba8af62 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:10.368998 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1377db08-85eb-4023-bd63-8a7ebba8af62 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:10.369278 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1377db08-85eb-4023-bd63-8a7ebba8af62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:10.436094 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-1377db08-85eb-4023-bd63-8a7ebba8af62 None None] MFA Rules not processed for user `d36b38a6e2d34c41b827ff6db9742332`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:10.438158 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-1377db08-85eb-4023-bd63-8a7ebba8af62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:10.467863 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 861/1740] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:31:10 2022] POST /identity/v3/projects => generated 387 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:31:10.467863 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c038c-046f-41b0-9b5d-22140f380634 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:10.606791 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 880/1741] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:10 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:10.606952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c038c-046f-41b0-9b5d-22140f380634 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:10.607612 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1377db08-85eb-4023-bd63-8a7ebba8af62 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:10.623682 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82698ccd-24f0-408f-98cc-63d933f158e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:10.624553 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82698ccd-24f0-408f-98cc-63d933f158e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:10.624949 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82698ccd-24f0-408f-98cc-63d933f158e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:10.625335 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82698ccd-24f0-408f-98cc-63d933f158e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:10.626124 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82698ccd-24f0-408f-98cc-63d933f158e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:10.628795 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82698ccd-24f0-408f-98cc-63d933f158e3 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:10.644618 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1377db08-85eb-4023-bd63-8a7ebba8af62 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:10.657338 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff05a960-3d7b-4d59-a747-801bf6cbbf15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:10.663115 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05a960-3d7b-4d59-a747-801bf6cbbf15 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:10.663115 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05a960-3d7b-4d59-a747-801bf6cbbf15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:10.663306 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05a960-3d7b-4d59-a747-801bf6cbbf15 admin admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:10.664013 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff05a960-3d7b-4d59-a747-801bf6cbbf15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:10.666176 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff05a960-3d7b-4d59-a747-801bf6cbbf15 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:10.797500 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 881/1742] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:31:10 2022] POST /identity/v3/users => generated 294 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:31:10.798181 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff05a960-3d7b-4d59-a747-801bf6cbbf15 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:10.803043 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-ff05a960-3d7b-4d59-a747-801bf6cbbf15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:10.803920 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 862/1743] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:31:10 2022] POST /identity/v3/projects => generated 405 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:31:10.805444 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82698ccd-24f0-408f-98cc-63d933f158e3 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:10.973691 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82698ccd-24f0-408f-98cc-63d933f158e3 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:10.986412 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff05a960-3d7b-4d59-a747-801bf6cbbf15 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:10.990802 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4686ca17-e39f-41f2-8eba-9f6965446bf6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:10.991478 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686ca17-e39f-41f2-8eba-9f6965446bf6 service nova] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:10.991772 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686ca17-e39f-41f2-8eba-9f6965446bf6 service nova] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:10.992349 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686ca17-e39f-41f2-8eba-9f6965446bf6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:11.057625 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:11.104862 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4686ca17-e39f-41f2-8eba-9f6965446bf6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:11.107040 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4686ca17-e39f-41f2-8eba-9f6965446bf6 service nova] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:11.119193 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a266b5-eca6-4fba-a8dd-6c9c262a26a1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85154ce7446245738edcc8702dbf6861', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:11.121074 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a266b5-eca6-4fba-a8dd-6c9c262a26a1 service swift] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:11.121599 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a266b5-eca6-4fba-a8dd-6c9c262a26a1 service swift] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:11.122203 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a266b5-eca6-4fba-a8dd-6c9c262a26a1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:11.134039 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 863/1744] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:31:10 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:31:11.137397 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4686ca17-e39f-41f2-8eba-9f6965446bf6 service nova] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:11.140878 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-4686ca17-e39f-41f2-8eba-9f6965446bf6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:11.144557 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:11.289392 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a266b5-eca6-4fba-a8dd-6c9c262a26a1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:11.294221 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a266b5-eca6-4fba-a8dd-6c9c262a26a1 service swift] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:11.378096 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 882/1745] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 5 15:31:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 15:31:11.384733 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a266b5-eca6-4fba-a8dd-6c9c262a26a1 service swift] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:11.400136 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4686ca17-e39f-41f2-8eba-9f6965446bf6 service nova] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:11.413508 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca486f-723d-4063-b6ae-712a7f59757a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:11.414344 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca486f-723d-4063-b6ae-712a7f59757a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:11.414879 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca486f-723d-4063-b6ae-712a7f59757a admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:11.415920 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca486f-723d-4063-b6ae-712a7f59757a admin admin] PATH_INFO: `/v3/projects/6427bd29e4ed4a9f9398dabf662285fe` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:11.422964 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ca486f-723d-4063-b6ae-712a7f59757a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6427bd29e4ed4a9f9398dabf662285fe)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:11.422964 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ca486f-723d-4063-b6ae-712a7f59757a admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:11.444387 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a266b5-eca6-4fba-a8dd-6c9c262a26a1 service swift] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:11.475879 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330a0bbe-cfbb-4feb-81e9-4878fe311efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:11.476822 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a0bbe-cfbb-4feb-81e9-4878fe311efd admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:11.486121 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a0bbe-cfbb-4feb-81e9-4878fe311efd admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:11.486121 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a0bbe-cfbb-4feb-81e9-4878fe311efd admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:11.486121 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330a0bbe-cfbb-4feb-81e9-4878fe311efd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:11.491696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330a0bbe-cfbb-4feb-81e9-4878fe311efd admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:11.564105 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 883/1746] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:31:11.565620 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330a0bbe-cfbb-4feb-81e9-4878fe311efd admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:11.626079 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-66ca486f-723d-4063-b6ae-712a7f59757a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6427bd29e4ed4a9f9398dabf662285fe', 'request_id': 'req-66ca486f-723d-4063-b6ae-712a7f59757a'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:11.628802 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330a0bbe-cfbb-4feb-81e9-4878fe311efd admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:11.664183 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-66ca486f-723d-4063-b6ae-712a7f59757a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6427bd29e4ed4a9f9398dabf662285fe', 'request_id': 'req-66ca486f-723d-4063-b6ae-712a7f59757a'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:11.747299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-66ca486f-723d-4063-b6ae-712a7f59757a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6427bd29e4ed4a9f9398dabf662285fe', 'request_id': 'req-66ca486f-723d-4063-b6ae-712a7f59757a'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:11.778089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-66ca486f-723d-4063-b6ae-712a7f59757a admin admin] The token cache is being invalidate because project 6427bd29e4ed4a9f9398dabf662285fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:31:11.780034 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-66ca486f-723d-4063-b6ae-712a7f59757a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:11.798390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-66ca486f-723d-4063-b6ae-712a7f59757a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:11.798390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 864/1747] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:31:11 2022] DELETE /identity/v3/projects/6427bd29e4ed4a9f9398dabf662285fe => generated 0 bytes in 657 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:11.798390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca486f-723d-4063-b6ae-712a7f59757a admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:11.834572 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-66ca486f-723d-4063-b6ae-712a7f59757a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:11.860161 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e81db28-d144-4a83-9b4c-f6fa8d7a9dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:11.860985 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e81db28-d144-4a83-9b4c-f6fa8d7a9dbe admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:11.861503 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e81db28-d144-4a83-9b4c-f6fa8d7a9dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:11.863253 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e81db28-d144-4a83-9b4c-f6fa8d7a9dbe admin admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:11.867342 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e81db28-d144-4a83-9b4c-f6fa8d7a9dbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:11.867719 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e81db28-d144-4a83-9b4c-f6fa8d7a9dbe admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:11.912580 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca486f-723d-4063-b6ae-712a7f59757a admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:11.938377 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c9d8d2-be70-4a89-854f-1065502c7ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:11.939074 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9d8d2-be70-4a89-854f-1065502c7ac3 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:11.939368 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9d8d2-be70-4a89-854f-1065502c7ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:11.939742 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9d8d2-be70-4a89-854f-1065502c7ac3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:11.940536 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c9d8d2-be70-4a89-854f-1065502c7ac3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:11.952600 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c9d8d2-be70-4a89-854f-1065502c7ac3 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:11.984260 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 865/1748] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:31:12.003807 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca057996-3cec-4e04-ab5e-7eb29b27f8e8 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:12.009412 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca057996-3cec-4e04-ab5e-7eb29b27f8e8 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:12.009759 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca057996-3cec-4e04-ab5e-7eb29b27f8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:12.014745 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 884/1749] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:31:11 2022] POST /identity/v3/users => generated 309 bytes in 450 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:31:12.016315 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e81db28-d144-4a83-9b4c-f6fa8d7a9dbe admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:12.027874 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:12.073281 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-ca057996-3cec-4e04-ab5e-7eb29b27f8e8 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:12.074613 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-ca057996-3cec-4e04-ab5e-7eb29b27f8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:12.084405 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e81db28-d144-4a83-9b4c-f6fa8d7a9dbe admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:12.094440 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 866/1750] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:12.094440 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca057996-3cec-4e04-ab5e-7eb29b27f8e8 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:12.098826 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-ca057996-3cec-4e04-ab5e-7eb29b27f8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:12.103921 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf2a31-c752-43dd-8e74-dee6c58f5bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:12.108123 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf2a31-c752-43dd-8e74-dee6c58f5bb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:12.108892 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf2a31-c752-43dd-8e74-dee6c58f5bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:12.111012 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf2a31-c752-43dd-8e74-dee6c58f5bb1 admin admin] PATH_INFO: `/v3/projects/7f522c6d50634355a39b1ed6401d1b66/users/af1b1682fddb4993b5e1484ee1158121/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:12.141302 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca057996-3cec-4e04-ab5e-7eb29b27f8e8 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:12.157445 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4f8d6-322f-47fd-8e53-9ecb72013a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:12.158098 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4f8d6-322f-47fd-8e53-9ecb72013a1e admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:12.159115 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4f8d6-322f-47fd-8e53-9ecb72013a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:12.159449 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4f8d6-322f-47fd-8e53-9ecb72013a1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:12.165800 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd4f8d6-322f-47fd-8e53-9ecb72013a1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:12.165800 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd4f8d6-322f-47fd-8e53-9ecb72013a1e admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:12.166200 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:12.181674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bf2a31-c752-43dd-8e74-dee6c58f5bb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f522c6d50634355a39b1ed6401d1b66, user_id=af1b1682fddb4993b5e1484ee1158121, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:12.187395 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 867/1751] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:31:12.189554 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bf2a31-c752-43dd-8e74-dee6c58f5bb1 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:12.197827 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e9f44-d588-4593-a80f-3ba6652e2cdf None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:12.201642 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e9f44-d588-4593-a80f-3ba6652e2cdf None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:12.202152 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e9f44-d588-4593-a80f-3ba6652e2cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:12.262716 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-380e9f44-d588-4593-a80f-3ba6652e2cdf None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:12.265104 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-380e9f44-d588-4593-a80f-3ba6652e2cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:12.293175 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 868/1752] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:12.295481 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e9f44-d588-4593-a80f-3ba6652e2cdf None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:12.318652 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e9f44-d588-4593-a80f-3ba6652e2cdf None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:12.328128 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bb03fe-92aa-41f4-8237-6bcf0e30eccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:12.329190 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb03fe-92aa-41f4-8237-6bcf0e30eccd admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:12.329699 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb03fe-92aa-41f4-8237-6bcf0e30eccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:12.331409 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb03fe-92aa-41f4-8237-6bcf0e30eccd admin admin] PATH_INFO: `/v3/projects/8bf072219f434680b52aad1586cab489/users/84511b192b534e759c0e3338580ec74d/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:12.347065 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 885/1753] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:31:12 2022] PUT /identity/v3/projects/7f522c6d50634355a39b1ed6401d1b66/users/af1b1682fddb4993b5e1484ee1158121/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:12.357542 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf2a31-c752-43dd-8e74-dee6c58f5bb1 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:12.408485 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bb03fe-92aa-41f4-8237-6bcf0e30eccd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bf072219f434680b52aad1586cab489, user_id=84511b192b534e759c0e3338580ec74d, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:12.418644 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bb03fe-92aa-41f4-8237-6bcf0e30eccd admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:12.508251 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf2a31-c752-43dd-8e74-dee6c58f5bb1 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:12.513969 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18505479-fed8-4f03-8ed7-12d5725b47de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:12.514836 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18505479-fed8-4f03-8ed7-12d5725b47de admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:12.515305 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18505479-fed8-4f03-8ed7-12d5725b47de admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:12.515773 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18505479-fed8-4f03-8ed7-12d5725b47de admin admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:12.517427 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18505479-fed8-4f03-8ed7-12d5725b47de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:31:12.518452 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18505479-fed8-4f03-8ed7-12d5725b47de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:12.520716 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18505479-fed8-4f03-8ed7-12d5725b47de admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:12.550013 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 886/1754] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:31:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:31:12.553993 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490beffd-de55-453d-91be-5fcab50b1ac7 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:12.554543 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490beffd-de55-453d-91be-5fcab50b1ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:12.555537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490beffd-de55-453d-91be-5fcab50b1ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:12.575874 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 869/1755] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:31:12 2022] PUT /identity/v3/projects/8bf072219f434680b52aad1586cab489/users/84511b192b534e759c0e3338580ec74d/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:12.577167 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bb03fe-92aa-41f4-8237-6bcf0e30eccd admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:12.630165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-490beffd-de55-453d-91be-5fcab50b1ac7 None None] MFA Rules not processed for user `af1b1682fddb4993b5e1484ee1158121`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:12.632392 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-490beffd-de55-453d-91be-5fcab50b1ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:12.763276 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bb03fe-92aa-41f4-8237-6bcf0e30eccd admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:12.776789 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e925ecb-a367-4b06-b9ae-fbd97c4baee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:12.776789 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e925ecb-a367-4b06-b9ae-fbd97c4baee7 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:12.776789 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e925ecb-a367-4b06-b9ae-fbd97c4baee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:12.776789 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e925ecb-a367-4b06-b9ae-fbd97c4baee7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:12.777201 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e925ecb-a367-4b06-b9ae-fbd97c4baee7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:12.779436 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e925ecb-a367-4b06-b9ae-fbd97c4baee7 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:12.782215 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 887/1756] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:12 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:12.796859 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490beffd-de55-453d-91be-5fcab50b1ac7 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:12.829426 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490beffd-de55-453d-91be-5fcab50b1ac7 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:12.838669 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a7cdf-92c5-4cb3-8685-0973dff0d30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:12.845841 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a7cdf-92c5-4cb3-8685-0973dff0d30f admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:12.846872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a7cdf-92c5-4cb3-8685-0973dff0d30f admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:12.847204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a7cdf-92c5-4cb3-8685-0973dff0d30f admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:12.848022 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903a7cdf-92c5-4cb3-8685-0973dff0d30f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:12.854746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903a7cdf-92c5-4cb3-8685-0973dff0d30f admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:12.903861 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 888/1757] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:31:12.903861 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a7cdf-92c5-4cb3-8685-0973dff0d30f admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:12.929660 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a7cdf-92c5-4cb3-8685-0973dff0d30f admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:12.953305 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cccf52-42d1-4b7c-940f-e438ab5020e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:12.954008 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cccf52-42d1-4b7c-940f-e438ab5020e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:12.954341 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cccf52-42d1-4b7c-940f-e438ab5020e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:12.954676 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cccf52-42d1-4b7c-940f-e438ab5020e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:12.955998 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cccf52-42d1-4b7c-940f-e438ab5020e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:12.959574 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 870/1758] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:31:12 2022] POST /identity/v3/projects => generated 369 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:31:12.960550 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e925ecb-a367-4b06-b9ae-fbd97c4baee7 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:12.995191 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cccf52-42d1-4b7c-940f-e438ab5020e1 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:13.132824 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:13.237178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e925ecb-a367-4b06-b9ae-fbd97c4baee7 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:13.237178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58cdda6-0e2e-43ef-9aa3-6c1860b7430c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:13.237178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58cdda6-0e2e-43ef-9aa3-6c1860b7430c admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:13.237178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58cdda6-0e2e-43ef-9aa3-6c1860b7430c admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:13.238885 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58cdda6-0e2e-43ef-9aa3-6c1860b7430c admin admin] PATH_INFO: `/v3/projects/8bf072219f434680b52aad1586cab489/users/84511b192b534e759c0e3338580ec74d/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:13.244294 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58cdda6-0e2e-43ef-9aa3-6c1860b7430c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bf072219f434680b52aad1586cab489, user_id=84511b192b534e759c0e3338580ec74d, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:13.255647 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58cdda6-0e2e-43ef-9aa3-6c1860b7430c admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:13.297561 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 871/1759] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:31:12 2022] PUT /identity/v3/projects/8bf072219f434680b52aad1586cab489/users/84511b192b534e759c0e3338580ec74d/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:13.298482 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58cdda6-0e2e-43ef-9aa3-6c1860b7430c admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:13.301060 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 889/1760] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:31:12 2022] POST /identity/v3/projects => generated 387 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:31:13.305419 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5ff1c-c0b7-40f5-b180-f6c7122658d5 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:13.305795 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5ff1c-c0b7-40f5-b180-f6c7122658d5 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:13.306151 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5ff1c-c0b7-40f5-b180-f6c7122658d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:13.354763 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-2fc5ff1c-c0b7-40f5-b180-f6c7122658d5 None None] MFA Rules not processed for user `d13a1638b2ca497fb02c71615559d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:13.362634 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:13.363647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-2fc5ff1c-c0b7-40f5-b180-f6c7122658d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:13.467177 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58cdda6-0e2e-43ef-9aa3-6c1860b7430c admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:13.599225 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 890/1761] 10.222.0.17 () {58 vars in 1015 bytes} [Fri Aug 5 15:31:13 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:13.603532 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b5a03b-09a3-49ee-ae54-8356369de47d None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:13.608405 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b5a03b-09a3-49ee-ae54-8356369de47d None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:13.608405 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b5a03b-09a3-49ee-ae54-8356369de47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:13.694336 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be447afa-dce4-4b1e-acfb-45d811347b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:13.695889 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be447afa-dce4-4b1e-acfb-45d811347b31 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:13.697400 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be447afa-dce4-4b1e-acfb-45d811347b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:13.698691 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be447afa-dce4-4b1e-acfb-45d811347b31 admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:13.700380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be447afa-dce4-4b1e-acfb-45d811347b31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:13.706170 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be447afa-dce4-4b1e-acfb-45d811347b31 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:13.747078 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-97b5a03b-09a3-49ee-ae54-8356369de47d None None] MFA Rules not processed for user `84511b192b534e759c0e3338580ec74d`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:13.749838 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-97b5a03b-09a3-49ee-ae54-8356369de47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:13.789923 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 872/1762] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:31:13 2022] POST /identity/v3/users => generated 285 bytes in 492 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:31:13.794142 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be447afa-dce4-4b1e-acfb-45d811347b31 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:13.835857 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be447afa-dce4-4b1e-acfb-45d811347b31 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:13.862675 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6009a39-1d87-4dca-8530-c1f568fa065b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:13.863683 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6009a39-1d87-4dca-8530-c1f568fa065b admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:13.864326 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6009a39-1d87-4dca-8530-c1f568fa065b admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:13.883006 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6009a39-1d87-4dca-8530-c1f568fa065b admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:13.912545 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:13.912737 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6009a39-1d87-4dca-8530-c1f568fa065b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:13.912737 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6009a39-1d87-4dca-8530-c1f568fa065b admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:13.942072 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 891/1763] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:13 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:13.942908 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b5a03b-09a3-49ee-ae54-8356369de47d None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:13.950875 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-97b5a03b-09a3-49ee-ae54-8356369de47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:13.992751 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b5a03b-09a3-49ee-ae54-8356369de47d None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:14.004336 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c265e-67ef-499b-8aad-4e513292b010 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:14.018906 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c265e-67ef-499b-8aad-4e513292b010 service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:14.019207 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c265e-67ef-499b-8aad-4e513292b010 service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:14.023501 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c265e-67ef-499b-8aad-4e513292b010 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:14.043502 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 873/1764] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:31:13 2022] POST /identity/v3/users => generated 300 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:31:14.053245 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6009a39-1d87-4dca-8530-c1f568fa065b admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:14.071349 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546c265e-67ef-499b-8aad-4e513292b010 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:14.075093 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546c265e-67ef-499b-8aad-4e513292b010 service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:14.080998 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6009a39-1d87-4dca-8530-c1f568fa065b admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:14.105391 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a960b3c-e52d-4b83-bb95-01e411c7b71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:14.105988 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 892/1765] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:31:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:31:14.109358 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c265e-67ef-499b-8aad-4e513292b010 service neutron] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:14.114123 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-546c265e-67ef-499b-8aad-4e513292b010 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:14.123793 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a960b3c-e52d-4b83-bb95-01e411c7b71e admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:14.123793 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a960b3c-e52d-4b83-bb95-01e411c7b71e admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:14.123793 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a960b3c-e52d-4b83-bb95-01e411c7b71e admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:14.123793 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a960b3c-e52d-4b83-bb95-01e411c7b71e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:14.123793 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a960b3c-e52d-4b83-bb95-01e411c7b71e admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:14.162726 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 874/1766] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:31:14.173023 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a960b3c-e52d-4b83-bb95-01e411c7b71e admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:14.224231 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a960b3c-e52d-4b83-bb95-01e411c7b71e admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:14.244257 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b807b-828e-426f-bd18-7ca6193d0333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:14.246073 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b807b-828e-426f-bd18-7ca6193d0333 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:14.246073 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b807b-828e-426f-bd18-7ca6193d0333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:14.246073 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b807b-828e-426f-bd18-7ca6193d0333 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:14.246383 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27b807b-828e-426f-bd18-7ca6193d0333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:14.247726 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27b807b-828e-426f-bd18-7ca6193d0333 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:14.272460 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c265e-67ef-499b-8aad-4e513292b010 service neutron] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:14.284146 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1662070-5999-4be4-b129-c8b8f87db8d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:14.285213 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1662070-5999-4be4-b129-c8b8f87db8d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:14.285780 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1662070-5999-4be4-b129-c8b8f87db8d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:14.286066 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1662070-5999-4be4-b129-c8b8f87db8d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:14.310168 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1662070-5999-4be4-b129-c8b8f87db8d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:14.310949 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1662070-5999-4be4-b129-c8b8f87db8d4 service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:14.316579 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 875/1767] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:31:14.321093 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b807b-828e-426f-bd18-7ca6193d0333 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:14.345488 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 893/1768] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:31:14 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:31:14.347697 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1662070-5999-4be4-b129-c8b8f87db8d4 service cinder] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:14.351162 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-a1662070-5999-4be4-b129-c8b8f87db8d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:14.355704 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b807b-828e-426f-bd18-7ca6193d0333 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:14.385985 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf1fdc-b394-4905-abaa-dcf12da3839e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:14.409780 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf1fdc-b394-4905-abaa-dcf12da3839e admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:14.409872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1662070-5999-4be4-b129-c8b8f87db8d4 service cinder] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:14.410356 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf1fdc-b394-4905-abaa-dcf12da3839e admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:14.410356 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf1fdc-b394-4905-abaa-dcf12da3839e admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:14.414667 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cf1fdc-b394-4905-abaa-dcf12da3839e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:14.418506 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf3b0f-a02b-4e64-8fbb-40d98a671129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:14.419224 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf3b0f-a02b-4e64-8fbb-40d98a671129 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:14.419658 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf3b0f-a02b-4e64-8fbb-40d98a671129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:14.420351 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf3b0f-a02b-4e64-8fbb-40d98a671129 admin admin] PATH_INFO: `/v3/users/bb797da54e734a93be8524b43b8c345f` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:14.421920 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cf1fdc-b394-4905-abaa-dcf12da3839e admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:14.438230 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcf3b0f-a02b-4e64-8fbb-40d98a671129 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb797da54e734a93be8524b43b8c345f)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:14.447897 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcf3b0f-a02b-4e64-8fbb-40d98a671129 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:14.476482 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 876/1769] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:31:14.479236 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf1fdc-b394-4905-abaa-dcf12da3839e admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:14.508141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf1fdc-b394-4905-abaa-dcf12da3839e admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:14.551117 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb16fe-a263-4760-a847-daac9bfff73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:14.551340 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb16fe-a263-4760-a847-daac9bfff73e admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:14.551590 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb16fe-a263-4760-a847-daac9bfff73e admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:14.551799 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb16fe-a263-4760-a847-daac9bfff73e admin admin] PATH_INFO: `/v3/projects/112d51b876a54a0687b6a4bb1581cc75/users/4e66fd30e06141c7a1458939f31ad7cf/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:14.601757 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeb16fe-a263-4760-a847-daac9bfff73e admin admin] RBAC: Authorizing `identity:create_grant(project_id=112d51b876a54a0687b6a4bb1581cc75, user_id=4e66fd30e06141c7a1458939f31ad7cf, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:14.607696 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeb16fe-a263-4760-a847-daac9bfff73e admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:14.706875 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 877/1770] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:31:14 2022] PUT /identity/v3/projects/112d51b876a54a0687b6a4bb1581cc75/users/4e66fd30e06141c7a1458939f31ad7cf/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:14.706875 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb16fe-a263-4760-a847-daac9bfff73e admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:14.725565 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:14.787771 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-dfcf3b0f-a02b-4e64-8fbb-40d98a671129 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb797da54e734a93be8524b43b8c345f'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:14.788656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-dfcf3b0f-a02b-4e64-8fbb-40d98a671129 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb797da54e734a93be8524b43b8c345f'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:14.838369 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-dfcf3b0f-a02b-4e64-8fbb-40d98a671129 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb797da54e734a93be8524b43b8c345f'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:14.875448 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb16fe-a263-4760-a847-daac9bfff73e admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:14.893258 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df347ed-368e-4be7-97fd-2bdb8114e9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:14.893964 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df347ed-368e-4be7-97fd-2bdb8114e9fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:14.894251 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df347ed-368e-4be7-97fd-2bdb8114e9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:14.894533 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df347ed-368e-4be7-97fd-2bdb8114e9fc admin admin] PATH_INFO: `/v3/projects/413e184357684d2db032404bfeeb098e/users/3181876ee88c4d218312bc3d7cc44502/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:14.926791 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df347ed-368e-4be7-97fd-2bdb8114e9fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=413e184357684d2db032404bfeeb098e, user_id=3181876ee88c4d218312bc3d7cc44502, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:14.929965 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df347ed-368e-4be7-97fd-2bdb8114e9fc admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:14.952272 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:14.960264 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-dfcf3b0f-a02b-4e64-8fbb-40d98a671129 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb797da54e734a93be8524b43b8c345f'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:14.972200 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-dfcf3b0f-a02b-4e64-8fbb-40d98a671129 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb797da54e734a93be8524b43b8c345f'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:14.989599 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 894/1771] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:31:14 2022] DELETE /identity/v3/users/bb797da54e734a93be8524b43b8c345f => generated 0 bytes in 643 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:14.993629 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf3b0f-a02b-4e64-8fbb-40d98a671129 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:14.995107 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-dfcf3b0f-a02b-4e64-8fbb-40d98a671129 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:15.057837 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 878/1772] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:31:14 2022] PUT /identity/v3/projects/413e184357684d2db032404bfeeb098e/users/3181876ee88c4d218312bc3d7cc44502/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:15.057837 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df347ed-368e-4be7-97fd-2bdb8114e9fc admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:15.057837 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-9df347ed-368e-4be7-97fd-2bdb8114e9fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:15.162615 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df347ed-368e-4be7-97fd-2bdb8114e9fc admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:15.168523 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d603ab-7e1e-4ee4-b69e-1cea957d83d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:15.171681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d603ab-7e1e-4ee4-b69e-1cea957d83d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:15.171681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d603ab-7e1e-4ee4-b69e-1cea957d83d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:15.172753 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d603ab-7e1e-4ee4-b69e-1cea957d83d5 admin admin] PATH_INFO: `/v3/users/ef1e3ac4feb245e3bb710d7dd43e5fb7` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:15.177959 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d603ab-7e1e-4ee4-b69e-1cea957d83d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef1e3ac4feb245e3bb710d7dd43e5fb7)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:15.181263 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d603ab-7e1e-4ee4-b69e-1cea957d83d5 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:15.270954 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf3b0f-a02b-4e64-8fbb-40d98a671129 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:15.299013 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f894bfd-dccc-469a-a766-d1b701cdd1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:15.299658 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f894bfd-dccc-469a-a766-d1b701cdd1ba admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:15.300614 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f894bfd-dccc-469a-a766-d1b701cdd1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:15.300614 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f894bfd-dccc-469a-a766-d1b701cdd1ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:15.301050 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f894bfd-dccc-469a-a766-d1b701cdd1ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:15.305431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f894bfd-dccc-469a-a766-d1b701cdd1ba admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:15.348953 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 895/1773] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:31:15.359471 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2485332-54df-4329-a846-d9c0fa13a9c9 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:15.360422 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2485332-54df-4329-a846-d9c0fa13a9c9 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:15.360828 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2485332-54df-4329-a846-d9c0fa13a9c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:15.405840 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-d2485332-54df-4329-a846-d9c0fa13a9c9 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:15.407292 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-d2485332-54df-4329-a846-d9c0fa13a9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:15.418465 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 896/1774] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:15.423023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b834ee20-b57a-45d4-86f1-e634ffb5c51d None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:15.423427 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b834ee20-b57a-45d4-86f1-e634ffb5c51d None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:15.423824 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b834ee20-b57a-45d4-86f1-e634ffb5c51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:15.431681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-97d603ab-7e1e-4ee4-b69e-1cea957d83d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef1e3ac4feb245e3bb710d7dd43e5fb7'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:15.451552 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-97d603ab-7e1e-4ee4-b69e-1cea957d83d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef1e3ac4feb245e3bb710d7dd43e5fb7'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:15.452529 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-97d603ab-7e1e-4ee4-b69e-1cea957d83d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef1e3ac4feb245e3bb710d7dd43e5fb7'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:15.475883 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-97d603ab-7e1e-4ee4-b69e-1cea957d83d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef1e3ac4feb245e3bb710d7dd43e5fb7'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:15.477058 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-97d603ab-7e1e-4ee4-b69e-1cea957d83d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef1e3ac4feb245e3bb710d7dd43e5fb7'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:15.483209 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-b834ee20-b57a-45d4-86f1-e634ffb5c51d None None] MFA Rules not processed for user `3181876ee88c4d218312bc3d7cc44502`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:15.485230 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-b834ee20-b57a-45d4-86f1-e634ffb5c51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:15.624765 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 879/1775] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:31:15 2022] DELETE /identity/v3/users/ef1e3ac4feb245e3bb710d7dd43e5fb7 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:15.626654 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d603ab-7e1e-4ee4-b69e-1cea957d83d5 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:15.632236 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-97d603ab-7e1e-4ee4-b69e-1cea957d83d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:15.661013 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 897/1776] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:15.662547 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b834ee20-b57a-45d4-86f1-e634ffb5c51d None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:15.666646 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-b834ee20-b57a-45d4-86f1-e634ffb5c51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:15.710414 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:15.771883 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b834ee20-b57a-45d4-86f1-e634ffb5c51d None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:15.833021 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d603ab-7e1e-4ee4-b69e-1cea957d83d5 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:15.859076 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268e5c83-df80-4dcb-a6cc-16d8391cd980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:15.860248 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e5c83-df80-4dcb-a6cc-16d8391cd980 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:15.860709 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e5c83-df80-4dcb-a6cc-16d8391cd980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:15.861122 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e5c83-df80-4dcb-a6cc-16d8391cd980 admin admin] PATH_INFO: `/v3/projects/112d51b876a54a0687b6a4bb1581cc75/users/4e66fd30e06141c7a1458939f31ad7cf/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:15.866585 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268e5c83-df80-4dcb-a6cc-16d8391cd980 admin admin] RBAC: Authorizing `identity:create_grant(project_id=112d51b876a54a0687b6a4bb1581cc75, user_id=4e66fd30e06141c7a1458939f31ad7cf, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:15.876909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268e5c83-df80-4dcb-a6cc-16d8391cd980 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:15.900014 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d5652-4812-48c5-93f5-d7fc5e951dcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:15.900687 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d5652-4812-48c5-93f5-d7fc5e951dcd service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:15.901083 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d5652-4812-48c5-93f5-d7fc5e951dcd service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:15.919708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d5652-4812-48c5-93f5-d7fc5e951dcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:15.925351 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-274d5652-4812-48c5-93f5-d7fc5e951dcd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:15.946434 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 880/1777] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:31:15 2022] PUT /identity/v3/projects/112d51b876a54a0687b6a4bb1581cc75/users/4e66fd30e06141c7a1458939f31ad7cf/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:15.946434 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31339b8-eb43-4786-83e2-515579351ebd None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:15.946726 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274d5652-4812-48c5-93f5-d7fc5e951dcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:15.948356 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31339b8-eb43-4786-83e2-515579351ebd None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:15.948710 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31339b8-eb43-4786-83e2-515579351ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:15.949039 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274d5652-4812-48c5-93f5-d7fc5e951dcd service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:15.962750 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 898/1778] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:31:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:31:15.963611 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d5652-4812-48c5-93f5-d7fc5e951dcd service neutron] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:15.965460 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-274d5652-4812-48c5-93f5-d7fc5e951dcd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:16.006415 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-e31339b8-eb43-4786-83e2-515579351ebd None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:16.007812 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-e31339b8-eb43-4786-83e2-515579351ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:16.009894 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d5652-4812-48c5-93f5-d7fc5e951dcd service neutron] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:16.011072 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:16.183398 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d0f45-c4dc-442d-88ee-3ff00f6e9679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:16.184762 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d0f45-c4dc-442d-88ee-3ff00f6e9679 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:16.185284 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d0f45-c4dc-442d-88ee-3ff00f6e9679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:16.185719 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d0f45-c4dc-442d-88ee-3ff00f6e9679 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:16.186657 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067d0f45-c4dc-442d-88ee-3ff00f6e9679 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:16.188883 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067d0f45-c4dc-442d-88ee-3ff00f6e9679 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:16.199438 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 881/1779] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:16.205692 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa17e1f7-a133-4757-8491-afcd86711f44 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:16.208254 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa17e1f7-a133-4757-8491-afcd86711f44 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:16.210253 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa17e1f7-a133-4757-8491-afcd86711f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:16.254970 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-aa17e1f7-a133-4757-8491-afcd86711f44 None None] MFA Rules not processed for user `4e66fd30e06141c7a1458939f31ad7cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:16.256627 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-aa17e1f7-a133-4757-8491-afcd86711f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:16.272483 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 899/1780] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:31:15 2022] POST /identity/v3/projects => generated 367 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:31:16.277140 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d0f45-c4dc-442d-88ee-3ff00f6e9679 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:16.377533 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d0f45-c4dc-442d-88ee-3ff00f6e9679 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:16.384455 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc544e41-8bc6-4f51-b2e2-9ab02a754346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:16.386928 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc544e41-8bc6-4f51-b2e2-9ab02a754346 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:16.386928 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc544e41-8bc6-4f51-b2e2-9ab02a754346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:16.386928 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc544e41-8bc6-4f51-b2e2-9ab02a754346 admin admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:16.386928 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc544e41-8bc6-4f51-b2e2-9ab02a754346 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:16.389730 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc544e41-8bc6-4f51-b2e2-9ab02a754346 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:16.415873 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 882/1781] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:16 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:16.421590 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa17e1f7-a133-4757-8491-afcd86711f44 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:16.422529 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-aa17e1f7-a133-4757-8491-afcd86711f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:16.487493 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 900/1782] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:31:16 2022] POST /identity/v3/users => generated 290 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:31:16.488477 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc544e41-8bc6-4f51-b2e2-9ab02a754346 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:16.493135 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-bc544e41-8bc6-4f51-b2e2-9ab02a754346 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:16.585792 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa17e1f7-a133-4757-8491-afcd86711f44 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:16.646999 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc544e41-8bc6-4f51-b2e2-9ab02a754346 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:16.673376 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441fb0fd-a17d-4e7b-8167-8df0262927a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:16.675226 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441fb0fd-a17d-4e7b-8167-8df0262927a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:16.675824 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441fb0fd-a17d-4e7b-8167-8df0262927a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:16.676566 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441fb0fd-a17d-4e7b-8167-8df0262927a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:16.718312 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c97d5-7196-441b-a879-d21c574ace64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:16.719063 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c97d5-7196-441b-a879-d21c574ace64 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:16.722879 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c97d5-7196-441b-a879-d21c574ace64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:16.724154 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c97d5-7196-441b-a879-d21c574ace64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:16.769618 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55c97d5-7196-441b-a879-d21c574ace64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:16.771535 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55c97d5-7196-441b-a879-d21c574ace64 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:16.811216 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 883/1783] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:31:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:31:16.816326 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c97d5-7196-441b-a879-d21c574ace64 service neutron] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:16.858217 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441fb0fd-a17d-4e7b-8167-8df0262927a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:16.860309 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441fb0fd-a17d-4e7b-8167-8df0262927a6 service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:16.866044 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c97d5-7196-441b-a879-d21c574ace64 service neutron] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:16.877558 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc73cab-70e0-44b3-8237-d5b399b63a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:16.879164 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc73cab-70e0-44b3-8237-d5b399b63a01 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:16.879449 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc73cab-70e0-44b3-8237-d5b399b63a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:16.879990 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc73cab-70e0-44b3-8237-d5b399b63a01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:16.880703 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc73cab-70e0-44b3-8237-d5b399b63a01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:16.884817 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 901/1784] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:31:16 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:31:16.890257 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc73cab-70e0-44b3-8237-d5b399b63a01 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:16.920099 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 884/1785] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:31:16.920475 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441fb0fd-a17d-4e7b-8167-8df0262927a6 service cinder] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:16.951532 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441fb0fd-a17d-4e7b-8167-8df0262927a6 service cinder] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:16.960588 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e4364c-e5ec-4500-bdbf-d4b835e0ad4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:16.961399 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4364c-e5ec-4500-bdbf-d4b835e0ad4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:16.961697 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4364c-e5ec-4500-bdbf-d4b835e0ad4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:16.962059 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4364c-e5ec-4500-bdbf-d4b835e0ad4e admin admin] PATH_INFO: `/v3/projects/4f3b25ec2aa84c60b10ae9d34757f1bb/users/ff6dee211eb642579972ee81767402f0/roles/7c88c815db854954b516ab97cd471647` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:16.996824 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc73cab-70e0-44b3-8237-d5b399b63a01 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:17.030669 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc73cab-70e0-44b3-8237-d5b399b63a01 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:17.036732 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:17.068017 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e4364c-e5ec-4500-bdbf-d4b835e0ad4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f3b25ec2aa84c60b10ae9d34757f1bb, user_id=ff6dee211eb642579972ee81767402f0, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:17.071339 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2475002f-b152-4403-8f50-e75f817c13e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:17.074034 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e4364c-e5ec-4500-bdbf-d4b835e0ad4e admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:17.074571 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475002f-b152-4403-8f50-e75f817c13e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:17.075247 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475002f-b152-4403-8f50-e75f817c13e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:17.076737 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475002f-b152-4403-8f50-e75f817c13e2 admin admin] PATH_INFO: `/v3/users/af1b1682fddb4993b5e1484ee1158121` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:17.080790 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2475002f-b152-4403-8f50-e75f817c13e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af1b1682fddb4993b5e1484ee1158121)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:17.087482 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2475002f-b152-4403-8f50-e75f817c13e2 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:17.177944 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:17.218440 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 902/1786] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:31:16 2022] PUT /identity/v3/projects/4f3b25ec2aa84c60b10ae9d34757f1bb/users/ff6dee211eb642579972ee81767402f0/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:17.222626 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e4364c-e5ec-4500-bdbf-d4b835e0ad4e admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:17.421436 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-2475002f-b152-4403-8f50-e75f817c13e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af1b1682fddb4993b5e1484ee1158121'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:17.444187 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-2475002f-b152-4403-8f50-e75f817c13e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af1b1682fddb4993b5e1484ee1158121'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:17.445328 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-2475002f-b152-4403-8f50-e75f817c13e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af1b1682fddb4993b5e1484ee1158121'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:17.476210 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-2475002f-b152-4403-8f50-e75f817c13e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af1b1682fddb4993b5e1484ee1158121'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:17.477782 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-2475002f-b152-4403-8f50-e75f817c13e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af1b1682fddb4993b5e1484ee1158121'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:17.502553 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e4364c-e5ec-4500-bdbf-d4b835e0ad4e admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:17.521283 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1382ef8-37b6-4415-ad2a-f00fa60df786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:17.522257 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1382ef8-37b6-4415-ad2a-f00fa60df786 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:17.522711 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1382ef8-37b6-4415-ad2a-f00fa60df786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:17.523160 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1382ef8-37b6-4415-ad2a-f00fa60df786 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:17.524299 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1382ef8-37b6-4415-ad2a-f00fa60df786 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:17.526320 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1382ef8-37b6-4415-ad2a-f00fa60df786 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:17.560478 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 903/1787] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:31:17.560935 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 885/1788] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:31:16 2022] DELETE /identity/v3/users/af1b1682fddb4993b5e1484ee1158121 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:17.565682 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1382ef8-37b6-4415-ad2a-f00fa60df786 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:17.567399 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f7daf-8177-4715-a1f4-44df71a8e06d None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:17.567848 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f7daf-8177-4715-a1f4-44df71a8e06d None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:17.568509 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f7daf-8177-4715-a1f4-44df71a8e06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:17.571512 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-e1382ef8-37b6-4415-ad2a-f00fa60df786 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:17.599445 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1382ef8-37b6-4415-ad2a-f00fa60df786 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:17.662190 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-737f7daf-8177-4715-a1f4-44df71a8e06d None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:17.666428 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-737f7daf-8177-4715-a1f4-44df71a8e06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:17.772734 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:17.788177 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601202de-f8e2-4778-b64b-8fa51588c967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:17.788587 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601202de-f8e2-4778-b64b-8fa51588c967 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:17.788883 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601202de-f8e2-4778-b64b-8fa51588c967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:17.789178 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601202de-f8e2-4778-b64b-8fa51588c967 admin admin] PATH_INFO: `/v3/projects/4f3b25ec2aa84c60b10ae9d34757f1bb/users/ff6dee211eb642579972ee81767402f0/roles/7c88c815db854954b516ab97cd471647` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:17.796712 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601202de-f8e2-4778-b64b-8fa51588c967 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f3b25ec2aa84c60b10ae9d34757f1bb, user_id=ff6dee211eb642579972ee81767402f0, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:17.810597 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601202de-f8e2-4778-b64b-8fa51588c967 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:17.857266 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 886/1790] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:17.863819 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 904/1790] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:31:17 2022] PUT /identity/v3/projects/4f3b25ec2aa84c60b10ae9d34757f1bb/users/ff6dee211eb642579972ee81767402f0/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:17.864498 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8ac0c-e050-4dde-b0ee-80fec66d5203 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:17.865122 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8ac0c-e050-4dde-b0ee-80fec66d5203 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:17.865995 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8ac0c-e050-4dde-b0ee-80fec66d5203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:17.927928 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-ebe8ac0c-e050-4dde-b0ee-80fec66d5203 None None] MFA Rules not processed for user `ff6dee211eb642579972ee81767402f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:17.929423 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-ebe8ac0c-e050-4dde-b0ee-80fec66d5203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:18.000144 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601202de-f8e2-4778-b64b-8fa51588c967 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:18.003965 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-601202de-f8e2-4778-b64b-8fa51588c967 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:18.087217 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:18.172427 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 887/1791] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:17 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:18.180431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601202de-f8e2-4778-b64b-8fa51588c967 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:18.188715 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe8ac0c-e050-4dde-b0ee-80fec66d5203 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:18.190590 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-ebe8ac0c-e050-4dde-b0ee-80fec66d5203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:18.209556 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c0fce4-ba8a-4e9e-aa99-123d97cec7b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:18.212980 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0fce4-ba8a-4e9e-aa99-123d97cec7b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:18.228733 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0fce4-ba8a-4e9e-aa99-123d97cec7b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:18.228733 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0fce4-ba8a-4e9e-aa99-123d97cec7b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:18.382735 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe8ac0c-e050-4dde-b0ee-80fec66d5203 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:18.390336 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe33e6ce-1292-4a33-8b49-a6c774e13c7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:18.391249 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33e6ce-1292-4a33-8b49-a6c774e13c7a service cinder] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:18.391590 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33e6ce-1292-4a33-8b49-a6c774e13c7a service cinder] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:18.393308 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33e6ce-1292-4a33-8b49-a6c774e13c7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:18.411682 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe33e6ce-1292-4a33-8b49-a6c774e13c7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:18.413555 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe33e6ce-1292-4a33-8b49-a6c774e13c7a service cinder] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:18.432294 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 888/1792] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:31:18 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 15:31:18.474159 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c0fce4-ba8a-4e9e-aa99-123d97cec7b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:18.480245 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c0fce4-ba8a-4e9e-aa99-123d97cec7b6 service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:18.501027 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 905/1793] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:31:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:31:18.541583 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe33e6ce-1292-4a33-8b49-a6c774e13c7a service cinder] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:18.544439 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-fe33e6ce-1292-4a33-8b49-a6c774e13c7a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:18.566688 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe33e6ce-1292-4a33-8b49-a6c774e13c7a service cinder] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:18.587797 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c0fce4-ba8a-4e9e-aa99-123d97cec7b6 service neutron] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:18.591611 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-36c0fce4-ba8a-4e9e-aa99-123d97cec7b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:18.597839 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed39c1-8da8-40f0-96e0-d9e1df494263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:18.599166 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed39c1-8da8-40f0-96e0-d9e1df494263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:18.600046 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed39c1-8da8-40f0-96e0-d9e1df494263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:18.600419 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed39c1-8da8-40f0-96e0-d9e1df494263 admin admin] PATH_INFO: `/v3/projects/f54498df9fc64213b1b50cfc558e3502` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:18.604555 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ed39c1-8da8-40f0-96e0-d9e1df494263 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f54498df9fc64213b1b50cfc558e3502)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:18.604555 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ed39c1-8da8-40f0-96e0-d9e1df494263 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:18.621614 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c0fce4-ba8a-4e9e-aa99-123d97cec7b6 service neutron] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:18.635351 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d004753a-c667-4b5c-998a-00633d8f121b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:18.636601 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004753a-c667-4b5c-998a-00633d8f121b service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:18.637368 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004753a-c667-4b5c-998a-00633d8f121b service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:18.637907 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004753a-c667-4b5c-998a-00633d8f121b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:18.643879 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-d004753a-c667-4b5c-998a-00633d8f121b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:18.706023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:18.793984 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d004753a-c667-4b5c-998a-00633d8f121b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:18.795679 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d004753a-c667-4b5c-998a-00633d8f121b service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:18.819331 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 906/1794] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:31:18 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 15:31:18.821129 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e26bf8-edc7-4379-989c-ce6a2929cd65 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:18.821129 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e26bf8-edc7-4379-989c-ce6a2929cd65 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:18.821369 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e26bf8-edc7-4379-989c-ce6a2929cd65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:18.854918 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-c2e26bf8-edc7-4379-989c-ce6a2929cd65 None None] MFA Rules not processed for user `04e80222936d4427909414a78e1dc489`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:18.857184 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-c2e26bf8-edc7-4379-989c-ce6a2929cd65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:18.903961 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-89ed39c1-8da8-40f0-96e0-d9e1df494263 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f54498df9fc64213b1b50cfc558e3502', 'request_id': 'req-89ed39c1-8da8-40f0-96e0-d9e1df494263'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:18.924547 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-89ed39c1-8da8-40f0-96e0-d9e1df494263 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f54498df9fc64213b1b50cfc558e3502', 'request_id': 'req-89ed39c1-8da8-40f0-96e0-d9e1df494263'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:19.022272 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 907/1795] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:19.023715 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300f223-0505-419f-a09d-ba9096b2737a None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:19.024034 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300f223-0505-419f-a09d-ba9096b2737a None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:19.024350 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300f223-0505-419f-a09d-ba9096b2737a None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:19.034734 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-89ed39c1-8da8-40f0-96e0-d9e1df494263 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f54498df9fc64213b1b50cfc558e3502', 'request_id': 'req-89ed39c1-8da8-40f0-96e0-d9e1df494263'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:19.046088 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:19.053429 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 908/1796] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 5 15:31:19 2022] GET /identity => generated 270 bytes in 31 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:31:19.059007 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-89ed39c1-8da8-40f0-96e0-d9e1df494263 admin admin] The token cache is being invalidate because project f54498df9fc64213b1b50cfc558e3502 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:31:19.059317 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-89ed39c1-8da8-40f0-96e0-d9e1df494263 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:19.063457 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-89ed39c1-8da8-40f0-96e0-d9e1df494263 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:19.074879 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 889/1797] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:31:18 2022] DELETE /identity/v3/projects/f54498df9fc64213b1b50cfc558e3502 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:19.074980 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300f223-0505-419f-a09d-ba9096b2737a None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:19.076674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-7300f223-0505-419f-a09d-ba9096b2737a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:19.274463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd8263-5dd1-4b9e-8739-ed962d280a01 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:19.275262 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd8263-5dd1-4b9e-8739-ed962d280a01 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:19.275714 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd8263-5dd1-4b9e-8739-ed962d280a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:19.289833 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300f223-0505-419f-a09d-ba9096b2737a None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:19.307872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae97add7-fa88-43b8-992a-a56d1ac00ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:19.308853 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97add7-fa88-43b8-992a-a56d1ac00ccd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:19.309292 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97add7-fa88-43b8-992a-a56d1ac00ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:19.309733 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97add7-fa88-43b8-992a-a56d1ac00ccd admin admin] PATH_INFO: `/v3/users/fd1776073693456d89a754852ef1c804` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:19.311139 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae97add7-fa88-43b8-992a-a56d1ac00ccd admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd1776073693456d89a754852ef1c804)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:19.313294 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae97add7-fa88-43b8-992a-a56d1ac00ccd admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:19.313761 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-b8bd8263-5dd1-4b9e-8739-ed962d280a01 None None] MFA Rules not processed for user `bc0497dab1f44bb7b8196d39bf01d770`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:19.316563 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-b8bd8263-5dd1-4b9e-8739-ed962d280a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:19.462146 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 890/1798] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 5 15:31:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:31:19.610138 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-ae97add7-fa88-43b8-992a-a56d1ac00ccd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd1776073693456d89a754852ef1c804'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:19.611237 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-ae97add7-fa88-43b8-992a-a56d1ac00ccd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd1776073693456d89a754852ef1c804'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:19.650740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-ae97add7-fa88-43b8-992a-a56d1ac00ccd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd1776073693456d89a754852ef1c804'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:19.760672 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-ae97add7-fa88-43b8-992a-a56d1ac00ccd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd1776073693456d89a754852ef1c804'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:19.770029 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-ae97add7-fa88-43b8-992a-a56d1ac00ccd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd1776073693456d89a754852ef1c804'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:19.770765 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:19.793951 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 909/1799] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:31:19 2022] DELETE /identity/v3/users/fd1776073693456d89a754852ef1c804 => generated 0 bytes in 716 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:19.870712 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd8263-5dd1-4b9e-8739-ed962d280a01 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:19.872554 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-b8bd8263-5dd1-4b9e-8739-ed962d280a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:19.888745 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae97add7-fa88-43b8-992a-a56d1ac00ccd admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:19.893310 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-ae97add7-fa88-43b8-992a-a56d1ac00ccd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:19.925216 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd8263-5dd1-4b9e-8739-ed962d280a01 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:20.075389 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f6c95-6725-460f-8025-048cf70bdf93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:20.078176 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f6c95-6725-460f-8025-048cf70bdf93 service nova] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:20.078455 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f6c95-6725-460f-8025-048cf70bdf93 service nova] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:20.080516 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f6c95-6725-460f-8025-048cf70bdf93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:20.083453 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-e33f6c95-6725-460f-8025-048cf70bdf93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:20.085649 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae97add7-fa88-43b8-992a-a56d1ac00ccd admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:20.092147 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4292ab4-8c1e-4411-80bf-9155255ee14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:20.101684 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4292ab4-8c1e-4411-80bf-9155255ee14a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:20.101684 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4292ab4-8c1e-4411-80bf-9155255ee14a admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:20.101684 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4292ab4-8c1e-4411-80bf-9155255ee14a admin admin] PATH_INFO: `/v3/projects/3fb76d2caec94893b1ecb7a9b0e96986` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:20.101684 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4292ab4-8c1e-4411-80bf-9155255ee14a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fb76d2caec94893b1ecb7a9b0e96986)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:20.101684 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4292ab4-8c1e-4411-80bf-9155255ee14a admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:20.102216 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33f6c95-6725-460f-8025-048cf70bdf93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:20.104215 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33f6c95-6725-460f-8025-048cf70bdf93 service nova] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:20.139902 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 891/1800] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:31:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:31:20.306409 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-e4292ab4-8c1e-4411-80bf-9155255ee14a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fb76d2caec94893b1ecb7a9b0e96986', 'request_id': 'req-e4292ab4-8c1e-4411-80bf-9155255ee14a'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:20.359301 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-e4292ab4-8c1e-4411-80bf-9155255ee14a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fb76d2caec94893b1ecb7a9b0e96986', 'request_id': 'req-e4292ab4-8c1e-4411-80bf-9155255ee14a'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:20.371295 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-e4292ab4-8c1e-4411-80bf-9155255ee14a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fb76d2caec94893b1ecb7a9b0e96986', 'request_id': 'req-e4292ab4-8c1e-4411-80bf-9155255ee14a'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:20.504150 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-e4292ab4-8c1e-4411-80bf-9155255ee14a admin admin] The token cache is being invalidate because project 3fb76d2caec94893b1ecb7a9b0e96986 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:31:20.504461 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-e4292ab4-8c1e-4411-80bf-9155255ee14a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:20.505244 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-e4292ab4-8c1e-4411-80bf-9155255ee14a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:20.507409 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 910/1801] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:31:19 2022] DELETE /identity/v3/projects/3fb76d2caec94893b1ecb7a9b0e96986 => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:20.511564 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f6c95-6725-460f-8025-048cf70bdf93 service nova] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:20.540252 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f6c95-6725-460f-8025-048cf70bdf93 service nova] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:20.639893 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198454d8-1288-4d3c-add1-6839e66070b7 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:20.639893 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198454d8-1288-4d3c-add1-6839e66070b7 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:20.639893 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198454d8-1288-4d3c-add1-6839e66070b7 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:20.639893 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 911/1802] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 5 15:31:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:31:20.662544 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce695c-5a13-468d-a80e-da77a2ea28d3 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:20.662851 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce695c-5a13-468d-a80e-da77a2ea28d3 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:20.663239 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce695c-5a13-468d-a80e-da77a2ea28d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:20.689076 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b754d-18bd-49a0-b117-8e2dbd9ebe51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:20.690380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b754d-18bd-49a0-b117-8e2dbd9ebe51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:20.690685 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b754d-18bd-49a0-b117-8e2dbd9ebe51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:20.691669 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b754d-18bd-49a0-b117-8e2dbd9ebe51 admin admin] PATH_INFO: `/v3/users/84511b192b534e759c0e3338580ec74d` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:20.695996 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89b754d-18bd-49a0-b117-8e2dbd9ebe51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84511b192b534e759c0e3338580ec74d)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:20.700693 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89b754d-18bd-49a0-b117-8e2dbd9ebe51 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:20.706521 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-dbce695c-5a13-468d-a80e-da77a2ea28d3 None None] MFA Rules not processed for user `bc0497dab1f44bb7b8196d39bf01d770`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:20.710041 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-dbce695c-5a13-468d-a80e-da77a2ea28d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:20.762380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:20.973565 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 912/1803] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 5 15:31:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:21.031688 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-d89b754d-18bd-49a0-b117-8e2dbd9ebe51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84511b192b534e759c0e3338580ec74d'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:21.043666 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-d89b754d-18bd-49a0-b117-8e2dbd9ebe51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84511b192b534e759c0e3338580ec74d'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:21.045424 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-d89b754d-18bd-49a0-b117-8e2dbd9ebe51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84511b192b534e759c0e3338580ec74d'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:21.088610 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-d89b754d-18bd-49a0-b117-8e2dbd9ebe51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84511b192b534e759c0e3338580ec74d'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:21.089029 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-d89b754d-18bd-49a0-b117-8e2dbd9ebe51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84511b192b534e759c0e3338580ec74d'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:21.238511 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 892/1804] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:31:20 2022] DELETE /identity/v3/users/84511b192b534e759c0e3338580ec74d => generated 0 bytes in 725 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:21.420981 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce695c-5a13-468d-a80e-da77a2ea28d3 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:21.422515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-dbce695c-5a13-468d-a80e-da77a2ea28d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:21.479510 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce695c-5a13-468d-a80e-da77a2ea28d3 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:21.503820 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:21.601832 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b754d-18bd-49a0-b117-8e2dbd9ebe51 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:21.603459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-d89b754d-18bd-49a0-b117-8e2dbd9ebe51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:21.642303 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5480a60e-cf84-4211-bee0-0353fa355b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:21.643326 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480a60e-cf84-4211-bee0-0353fa355b69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:21.644801 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480a60e-cf84-4211-bee0-0353fa355b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:21.646387 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480a60e-cf84-4211-bee0-0353fa355b69 admin admin] PATH_INFO: `/v3/projects/7f522c6d50634355a39b1ed6401d1b66` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:21.649893 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5480a60e-cf84-4211-bee0-0353fa355b69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f522c6d50634355a39b1ed6401d1b66)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:21.652481 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5480a60e-cf84-4211-bee0-0353fa355b69 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:21.781906 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b754d-18bd-49a0-b117-8e2dbd9ebe51 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:21.789282 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-5480a60e-cf84-4211-bee0-0353fa355b69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f522c6d50634355a39b1ed6401d1b66', 'request_id': 'req-5480a60e-cf84-4211-bee0-0353fa355b69'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:21.822053 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-5480a60e-cf84-4211-bee0-0353fa355b69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f522c6d50634355a39b1ed6401d1b66', 'request_id': 'req-5480a60e-cf84-4211-bee0-0353fa355b69'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:21.836146 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-5480a60e-cf84-4211-bee0-0353fa355b69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f522c6d50634355a39b1ed6401d1b66', 'request_id': 'req-5480a60e-cf84-4211-bee0-0353fa355b69'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:21.864629 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa24b80-2af6-4a5e-aae0-272f68cc1638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:21.867273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa24b80-2af6-4a5e-aae0-272f68cc1638 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:21.868816 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa24b80-2af6-4a5e-aae0-272f68cc1638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:21.869271 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa24b80-2af6-4a5e-aae0-272f68cc1638 admin admin] PATH_INFO: `/v3/projects/a4715fb4480b4119b8212f2f4b0c6063` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:21.872517 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa24b80-2af6-4a5e-aae0-272f68cc1638 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4715fb4480b4119b8212f2f4b0c6063)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:21.879639 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa24b80-2af6-4a5e-aae0-272f68cc1638 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:21.957823 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-5480a60e-cf84-4211-bee0-0353fa355b69 admin admin] The token cache is being invalidate because project 7f522c6d50634355a39b1ed6401d1b66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:31:21.960528 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-5480a60e-cf84-4211-bee0-0353fa355b69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:21.963159 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-5480a60e-cf84-4211-bee0-0353fa355b69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:21.966411 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 913/1805] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:31:21 2022] DELETE /identity/v3/projects/7f522c6d50634355a39b1ed6401d1b66 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:21.971831 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5480a60e-cf84-4211-bee0-0353fa355b69 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:21.973992 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-5480a60e-cf84-4211-bee0-0353fa355b69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:22.006168 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5480a60e-cf84-4211-bee0-0353fa355b69 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:22.015401 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1f164-30dd-4cad-88c0-74fe3e4cf3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:22.016958 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1f164-30dd-4cad-88c0-74fe3e4cf3e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:22.017911 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1f164-30dd-4cad-88c0-74fe3e4cf3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:22.019321 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1f164-30dd-4cad-88c0-74fe3e4cf3e5 admin admin] PATH_INFO: `/v3/users/4e66fd30e06141c7a1458939f31ad7cf` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:22.021214 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd1f164-30dd-4cad-88c0-74fe3e4cf3e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e66fd30e06141c7a1458939f31ad7cf)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:22.024590 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd1f164-30dd-4cad-88c0-74fe3e4cf3e5 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:22.073228 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:22.078833 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-efa24b80-2af6-4a5e-aae0-272f68cc1638 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4715fb4480b4119b8212f2f4b0c6063', 'request_id': 'req-efa24b80-2af6-4a5e-aae0-272f68cc1638'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:22.088449 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-efa24b80-2af6-4a5e-aae0-272f68cc1638 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4715fb4480b4119b8212f2f4b0c6063', 'request_id': 'req-efa24b80-2af6-4a5e-aae0-272f68cc1638'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:22.182756 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-efa24b80-2af6-4a5e-aae0-272f68cc1638 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4715fb4480b4119b8212f2f4b0c6063', 'request_id': 'req-efa24b80-2af6-4a5e-aae0-272f68cc1638'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:22.207569 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-efa24b80-2af6-4a5e-aae0-272f68cc1638 admin admin] The token cache is being invalidate because project a4715fb4480b4119b8212f2f4b0c6063 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:31:22.210125 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-efa24b80-2af6-4a5e-aae0-272f68cc1638 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:22.211216 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-efa24b80-2af6-4a5e-aae0-272f68cc1638 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:22.221811 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 893/1806] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:31:21 2022] DELETE /identity/v3/projects/a4715fb4480b4119b8212f2f4b0c6063 => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:22.221811 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee799ae-6757-4642-814b-bcf86c342323 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:22.224152 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee799ae-6757-4642-814b-bcf86c342323 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:22.225021 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee799ae-6757-4642-814b-bcf86c342323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:22.284293 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-7ee799ae-6757-4642-814b-bcf86c342323 None None] MFA Rules not processed for user `d36b38a6e2d34c41b827ff6db9742332`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:22.286267 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-7ee799ae-6757-4642-814b-bcf86c342323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:22.304665 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-ddd1f164-30dd-4cad-88c0-74fe3e4cf3e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e66fd30e06141c7a1458939f31ad7cf'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:22.306459 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-ddd1f164-30dd-4cad-88c0-74fe3e4cf3e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e66fd30e06141c7a1458939f31ad7cf'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:22.370735 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 894/1807] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:22 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:22.370735 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c676b-233b-48ae-a43e-697b74efe918 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:22.370735 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c676b-233b-48ae-a43e-697b74efe918 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:22.370735 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c676b-233b-48ae-a43e-697b74efe918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:22.371086 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-ddd1f164-30dd-4cad-88c0-74fe3e4cf3e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e66fd30e06141c7a1458939f31ad7cf'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:22.391886 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-9d7c676b-233b-48ae-a43e-697b74efe918 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:22.393737 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-9d7c676b-233b-48ae-a43e-697b74efe918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:22.472563 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-ddd1f164-30dd-4cad-88c0-74fe3e4cf3e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e66fd30e06141c7a1458939f31ad7cf'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:22.486020 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-ddd1f164-30dd-4cad-88c0-74fe3e4cf3e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e66fd30e06141c7a1458939f31ad7cf'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:22.504244 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 914/1808] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:31:21 2022] DELETE /identity/v3/users/4e66fd30e06141c7a1458939f31ad7cf => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:31:22.507047 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1f164-30dd-4cad-88c0-74fe3e4cf3e5 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:22.511760 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-ddd1f164-30dd-4cad-88c0-74fe3e4cf3e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:22.599650 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 895/1809] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:22.601007 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d2110-a412-469c-8a76-0a25d435c460 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:22.602746 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d2110-a412-469c-8a76-0a25d435c460 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:22.602746 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d2110-a412-469c-8a76-0a25d435c460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:22.639725 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-286d2110-a412-469c-8a76-0a25d435c460 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:22.641425 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-286d2110-a412-469c-8a76-0a25d435c460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:22.738763 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1f164-30dd-4cad-88c0-74fe3e4cf3e5 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:22.747296 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa69965-ff1c-4e08-9b86-82fbc996bc95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:22.748219 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa69965-ff1c-4e08-9b86-82fbc996bc95 service nova] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:22.748581 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa69965-ff1c-4e08-9b86-82fbc996bc95 service nova] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:22.748963 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa69965-ff1c-4e08-9b86-82fbc996bc95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:22.751108 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-0aa69965-ff1c-4e08-9b86-82fbc996bc95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:22.757783 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 896/1810] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:22.761026 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286d2110-a412-469c-8a76-0a25d435c460 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:22.769473 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa69965-ff1c-4e08-9b86-82fbc996bc95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:22.772755 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa69965-ff1c-4e08-9b86-82fbc996bc95 service nova] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:22.791923 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 915/1811] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:31:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:31:22.801756 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286d2110-a412-469c-8a76-0a25d435c460 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:22.825611 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510fcf82-424a-48e2-8040-bb04705acf20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:22.826205 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510fcf82-424a-48e2-8040-bb04705acf20 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:22.826499 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510fcf82-424a-48e2-8040-bb04705acf20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:22.826784 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510fcf82-424a-48e2-8040-bb04705acf20 admin admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:22.827493 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510fcf82-424a-48e2-8040-bb04705acf20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:31:22.827808 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510fcf82-424a-48e2-8040-bb04705acf20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:22.829688 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510fcf82-424a-48e2-8040-bb04705acf20 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:22.867114 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 897/1812] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:31:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:31:22.873625 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa69965-ff1c-4e08-9b86-82fbc996bc95 service nova] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:22.900008 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa69965-ff1c-4e08-9b86-82fbc996bc95 service nova] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:22.917105 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b035a72-c849-43fb-b672-24fac11c46d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:22.918150 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b035a72-c849-43fb-b672-24fac11c46d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:22.918576 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b035a72-c849-43fb-b672-24fac11c46d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:22.919046 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b035a72-c849-43fb-b672-24fac11c46d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:22.919896 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b035a72-c849-43fb-b672-24fac11c46d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:22.924635 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b035a72-c849-43fb-b672-24fac11c46d1 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:22.947839 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:23.150373 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 916/1813] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:31:22 2022] POST /identity/v3/projects => generated 357 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:31:23.154566 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510fcf82-424a-48e2-8040-bb04705acf20 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:23.167954 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510fcf82-424a-48e2-8040-bb04705acf20 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:23.286417 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e508556a-391a-4617-aa19-18f9a35923cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:23.288572 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508556a-391a-4617-aa19-18f9a35923cc admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:23.289140 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508556a-391a-4617-aa19-18f9a35923cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:23.289642 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508556a-391a-4617-aa19-18f9a35923cc admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:23.290587 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e508556a-391a-4617-aa19-18f9a35923cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:23.293024 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e508556a-391a-4617-aa19-18f9a35923cc admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:23.415913 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 898/1814] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:31:23 2022] POST /identity/v3/users => generated 279 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:31:23.417010 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b035a72-c849-43fb-b672-24fac11c46d1 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:23.448298 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b035a72-c849-43fb-b672-24fac11c46d1 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:23.464885 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fecbbdd-c960-405c-ab8c-57fd3733eff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:23.465612 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fecbbdd-c960-405c-ab8c-57fd3733eff8 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:23.466005 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fecbbdd-c960-405c-ab8c-57fd3733eff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:23.466924 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fecbbdd-c960-405c-ab8c-57fd3733eff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:23.467662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fecbbdd-c960-405c-ab8c-57fd3733eff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:23.470073 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fecbbdd-c960-405c-ab8c-57fd3733eff8 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:23.497872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 917/1815] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:31:23 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 15:31:23.505325 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e508556a-391a-4617-aa19-18f9a35923cc admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:23.526748 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e508556a-391a-4617-aa19-18f9a35923cc admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:23.541085 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66742a6c-56ce-4391-b5ba-6c672bdc0fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:23.541839 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66742a6c-56ce-4391-b5ba-6c672bdc0fa2 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:23.542169 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66742a6c-56ce-4391-b5ba-6c672bdc0fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:23.542522 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66742a6c-56ce-4391-b5ba-6c672bdc0fa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:23.543328 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66742a6c-56ce-4391-b5ba-6c672bdc0fa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:23.545116 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66742a6c-56ce-4391-b5ba-6c672bdc0fa2 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:23.576183 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 899/1816] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:31:23 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:31:23.578193 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fecbbdd-c960-405c-ab8c-57fd3733eff8 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:23.606315 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fecbbdd-c960-405c-ab8c-57fd3733eff8 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:23.620573 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba027d90-cac7-474b-837e-f7cacfe8c1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:23.621668 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba027d90-cac7-474b-837e-f7cacfe8c1b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:23.643352 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba027d90-cac7-474b-837e-f7cacfe8c1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:23.643934 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba027d90-cac7-474b-837e-f7cacfe8c1b8 admin admin] PATH_INFO: `/v3/projects/bb8e2bf04bf24c24b37ddfc80217a84e/users/29bd7f872bed4bf280ae07c4bf82753b/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:23.648987 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66742a6c-56ce-4391-b5ba-6c672bdc0fa2 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:23.651444 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-66742a6c-56ce-4391-b5ba-6c672bdc0fa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:23.679762 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66742a6c-56ce-4391-b5ba-6c672bdc0fa2 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:23.715245 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba027d90-cac7-474b-837e-f7cacfe8c1b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb8e2bf04bf24c24b37ddfc80217a84e, user_id=29bd7f872bed4bf280ae07c4bf82753b, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:23.716006 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb1f178-5a91-4c1b-bdef-bc6bd1a138e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:23.717103 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb1f178-5a91-4c1b-bdef-bc6bd1a138e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:23.720231 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb1f178-5a91-4c1b-bdef-bc6bd1a138e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:23.722473 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba027d90-cac7-474b-837e-f7cacfe8c1b8 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:23.723442 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb1f178-5a91-4c1b-bdef-bc6bd1a138e6 admin admin] PATH_INFO: `/v3/projects/8bf072219f434680b52aad1586cab489` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:23.726270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb1f178-5a91-4c1b-bdef-bc6bd1a138e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bf072219f434680b52aad1586cab489)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:23.738460 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb1f178-5a91-4c1b-bdef-bc6bd1a138e6 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:23.743407 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:23.836867 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 918/1817] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:31:23 2022] PUT /identity/v3/projects/bb8e2bf04bf24c24b37ddfc80217a84e/users/29bd7f872bed4bf280ae07c4bf82753b/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:23.839052 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253eac53-a341-4c6b-a503-c30c64bb27de None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:23.839506 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253eac53-a341-4c6b-a503-c30c64bb27de None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:23.855250 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253eac53-a341-4c6b-a503-c30c64bb27de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:23.982000 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-253eac53-a341-4c6b-a503-c30c64bb27de None None] MFA Rules not processed for user `29bd7f872bed4bf280ae07c4bf82753b`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:23.986393 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-253eac53-a341-4c6b-a503-c30c64bb27de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:23.997130 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-abb1f178-5a91-4c1b-bdef-bc6bd1a138e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bf072219f434680b52aad1586cab489', 'request_id': 'req-abb1f178-5a91-4c1b-bdef-bc6bd1a138e6'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:24.025215 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-abb1f178-5a91-4c1b-bdef-bc6bd1a138e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bf072219f434680b52aad1586cab489', 'request_id': 'req-abb1f178-5a91-4c1b-bdef-bc6bd1a138e6'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:24.176932 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-abb1f178-5a91-4c1b-bdef-bc6bd1a138e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bf072219f434680b52aad1586cab489', 'request_id': 'req-abb1f178-5a91-4c1b-bdef-bc6bd1a138e6'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:24.190840 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 919/1818] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:23 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:24.195814 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253eac53-a341-4c6b-a503-c30c64bb27de None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:24.196982 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-abb1f178-5a91-4c1b-bdef-bc6bd1a138e6 admin admin] The token cache is being invalidate because project 8bf072219f434680b52aad1586cab489 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:31:24.197590 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-abb1f178-5a91-4c1b-bdef-bc6bd1a138e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:24.198420 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-abb1f178-5a91-4c1b-bdef-bc6bd1a138e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:24.203674 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 900/1819] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:31:23 2022] DELETE /identity/v3/projects/8bf072219f434680b52aad1586cab489 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:24.214312 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c6db5-4579-4063-89cf-a8a01ac39e77 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:24.215297 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c6db5-4579-4063-89cf-a8a01ac39e77 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:24.216055 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c6db5-4579-4063-89cf-a8a01ac39e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:24.275565 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-547c6db5-4579-4063-89cf-a8a01ac39e77 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:24.278555 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-547c6db5-4579-4063-89cf-a8a01ac39e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:24.396630 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 901/1820] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:24.402093 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50715b-57e8-469b-9a27-0886b46e3a61 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:24.404385 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50715b-57e8-469b-9a27-0886b46e3a61 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:24.404649 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253eac53-a341-4c6b-a503-c30c64bb27de None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:24.405898 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50715b-57e8-469b-9a27-0886b46e3a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:24.417844 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c26972-5a24-4a03-b47e-c1b844e7e7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:24.418722 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c26972-5a24-4a03-b47e-c1b844e7e7dc admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:24.420480 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c26972-5a24-4a03-b47e-c1b844e7e7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:24.420846 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c26972-5a24-4a03-b47e-c1b844e7e7dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:24.422038 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c26972-5a24-4a03-b47e-c1b844e7e7dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:24.430112 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c26972-5a24-4a03-b47e-c1b844e7e7dc admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:24.454062 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-de50715b-57e8-469b-9a27-0886b46e3a61 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:24.457685 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-de50715b-57e8-469b-9a27-0886b46e3a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:24.499690 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 902/1821] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:24.502476 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de50715b-57e8-469b-9a27-0886b46e3a61 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:24.506435 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-de50715b-57e8-469b-9a27-0886b46e3a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:24.533299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de50715b-57e8-469b-9a27-0886b46e3a61 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:24.564679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec7c73-6e04-4b36-810b-be7630229e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:24.565998 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec7c73-6e04-4b36-810b-be7630229e53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:24.565998 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec7c73-6e04-4b36-810b-be7630229e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:24.570470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec7c73-6e04-4b36-810b-be7630229e53 admin admin] PATH_INFO: `/v3/projects/112d51b876a54a0687b6a4bb1581cc75` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:24.570470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceec7c73-6e04-4b36-810b-be7630229e53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=112d51b876a54a0687b6a4bb1581cc75)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:24.582408 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceec7c73-6e04-4b36-810b-be7630229e53 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:24.599607 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 920/1822] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:31:24 2022] POST /identity/v3/projects => generated 355 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:31:24.603398 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c26972-5a24-4a03-b47e-c1b844e7e7dc admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:24.725052 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c26972-5a24-4a03-b47e-c1b844e7e7dc admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:24.731850 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8150017c-0ea0-49dc-996b-b30181e4aca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:24.732646 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150017c-0ea0-49dc-996b-b30181e4aca7 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:24.733260 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150017c-0ea0-49dc-996b-b30181e4aca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:24.734602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150017c-0ea0-49dc-996b-b30181e4aca7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:24.734602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8150017c-0ea0-49dc-996b-b30181e4aca7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:31:24.735036 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8150017c-0ea0-49dc-996b-b30181e4aca7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:24.737010 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8150017c-0ea0-49dc-996b-b30181e4aca7 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:24.768267 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 921/1823] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:31:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:31:24.769497 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8150017c-0ea0-49dc-996b-b30181e4aca7 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:24.771934 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-8150017c-0ea0-49dc-996b-b30181e4aca7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:24.786897 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8150017c-0ea0-49dc-996b-b30181e4aca7 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:24.797216 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-ceec7c73-6e04-4b36-810b-be7630229e53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '112d51b876a54a0687b6a4bb1581cc75', 'request_id': 'req-ceec7c73-6e04-4b36-810b-be7630229e53'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:24.804346 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-ceec7c73-6e04-4b36-810b-be7630229e53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '112d51b876a54a0687b6a4bb1581cc75', 'request_id': 'req-ceec7c73-6e04-4b36-810b-be7630229e53'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:24.875867 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7cddc0-e97e-4d95-99e8-518f80e3b2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:24.890238 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7cddc0-e97e-4d95-99e8-518f80e3b2a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:24.890721 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7cddc0-e97e-4d95-99e8-518f80e3b2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:24.891222 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7cddc0-e97e-4d95-99e8-518f80e3b2a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:24.894444 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7cddc0-e97e-4d95-99e8-518f80e3b2a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:24.896896 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7cddc0-e97e-4d95-99e8-518f80e3b2a2 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:24.913242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-ceec7c73-6e04-4b36-810b-be7630229e53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '112d51b876a54a0687b6a4bb1581cc75', 'request_id': 'req-ceec7c73-6e04-4b36-810b-be7630229e53'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:24.919529 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:24.936161 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-ceec7c73-6e04-4b36-810b-be7630229e53 admin admin] The token cache is being invalidate because project 112d51b876a54a0687b6a4bb1581cc75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:31:24.936758 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-ceec7c73-6e04-4b36-810b-be7630229e53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:24.937979 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-ceec7c73-6e04-4b36-810b-be7630229e53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:24.940867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 903/1824] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:31:24 2022] DELETE /identity/v3/projects/112d51b876a54a0687b6a4bb1581cc75 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:24.946108 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec7c73-6e04-4b36-810b-be7630229e53 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:24.947982 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-ceec7c73-6e04-4b36-810b-be7630229e53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:24.964717 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec7c73-6e04-4b36-810b-be7630229e53 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:24.970006 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b561fe8c-802f-4982-8e64-5bf338458a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:24.970620 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b561fe8c-802f-4982-8e64-5bf338458a62 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:24.975952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b561fe8c-802f-4982-8e64-5bf338458a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:24.975952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b561fe8c-802f-4982-8e64-5bf338458a62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:24.975952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b561fe8c-802f-4982-8e64-5bf338458a62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:24.975952 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b561fe8c-802f-4982-8e64-5bf338458a62 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:25.035927 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 922/1825] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:31:24 2022] POST /identity/v3/users => generated 284 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:31:25.037361 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d8002-fb5a-41ef-9fb7-e2def16776e7 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:25.038777 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d8002-fb5a-41ef-9fb7-e2def16776e7 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:25.039077 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d8002-fb5a-41ef-9fb7-e2def16776e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:25.118013 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-462d8002-fb5a-41ef-9fb7-e2def16776e7 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:25.120503 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-462d8002-fb5a-41ef-9fb7-e2def16776e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:25.121418 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 904/1826] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:31:24 2022] POST /identity/v3/projects => generated 371 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:31:25.126015 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b561fe8c-802f-4982-8e64-5bf338458a62 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:25.132204 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-b561fe8c-802f-4982-8e64-5bf338458a62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:25.239029 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b561fe8c-802f-4982-8e64-5bf338458a62 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:25.268421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3045c372-af9b-4428-b808-76e8c3ef1239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:25.270143 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045c372-af9b-4428-b808-76e8c3ef1239 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:25.270574 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045c372-af9b-4428-b808-76e8c3ef1239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:25.271039 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045c372-af9b-4428-b808-76e8c3ef1239 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:25.272089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3045c372-af9b-4428-b808-76e8c3ef1239 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:25.273960 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3045c372-af9b-4428-b808-76e8c3ef1239 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:25.279826 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 923/1827] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:25.281026 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d8002-fb5a-41ef-9fb7-e2def16776e7 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:25.296057 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d8002-fb5a-41ef-9fb7-e2def16776e7 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:25.302333 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815b8f63-9be9-4966-a36e-ec94013f52ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:25.303897 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b8f63-9be9-4966-a36e-ec94013f52ce admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:25.304387 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b8f63-9be9-4966-a36e-ec94013f52ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:25.304827 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b8f63-9be9-4966-a36e-ec94013f52ce admin admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:25.305702 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815b8f63-9be9-4966-a36e-ec94013f52ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:25.308943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815b8f63-9be9-4966-a36e-ec94013f52ce admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:25.310391 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 905/1828] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:31:25.312160 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5f1c9-a33d-47bf-b01d-4b1eb980b7c5 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:25.312505 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5f1c9-a33d-47bf-b01d-4b1eb980b7c5 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:25.312830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5f1c9-a33d-47bf-b01d-4b1eb980b7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:25.342207 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-abc5f1c9-a33d-47bf-b01d-4b1eb980b7c5 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:25.343423 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-abc5f1c9-a33d-47bf-b01d-4b1eb980b7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:25.353824 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 906/1829] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:25.354658 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5f1c9-a33d-47bf-b01d-4b1eb980b7c5 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:25.367373 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5f1c9-a33d-47bf-b01d-4b1eb980b7c5 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:25.376404 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fdc625-d3dd-4d7c-96f5-cf17bc48dd1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:25.376404 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fdc625-d3dd-4d7c-96f5-cf17bc48dd1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:25.376404 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fdc625-d3dd-4d7c-96f5-cf17bc48dd1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:25.376404 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fdc625-d3dd-4d7c-96f5-cf17bc48dd1a admin admin] PATH_INFO: `/v3/projects/87ce220cb82e438183b3483fc3917544/users/7a6efb71b0e74aaa86c6fb4993970bf8/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:25.400984 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fdc625-d3dd-4d7c-96f5-cf17bc48dd1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=87ce220cb82e438183b3483fc3917544, user_id=7a6efb71b0e74aaa86c6fb4993970bf8, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:25.403987 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fdc625-d3dd-4d7c-96f5-cf17bc48dd1a admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:25.433336 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 924/1830] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:31:25 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:31:25.434823 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815b8f63-9be9-4966-a36e-ec94013f52ce admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:25.468507 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815b8f63-9be9-4966-a36e-ec94013f52ce admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:25.468507 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b623ed10-a956-4830-af2b-7193ceabc123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:25.468507 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623ed10-a956-4830-af2b-7193ceabc123 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:25.468776 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623ed10-a956-4830-af2b-7193ceabc123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:25.468856 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623ed10-a956-4830-af2b-7193ceabc123 admin admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:25.492596 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b623ed10-a956-4830-af2b-7193ceabc123 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:31:25.499560 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 907/1831] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:31:25 2022] PUT /identity/v3/projects/87ce220cb82e438183b3483fc3917544/users/7a6efb71b0e74aaa86c6fb4993970bf8/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:25.500979 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fdc625-d3dd-4d7c-96f5-cf17bc48dd1a admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:25.501609 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b623ed10-a956-4830-af2b-7193ceabc123 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:25.505915 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b623ed10-a956-4830-af2b-7193ceabc123 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:25.546610 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 925/1832] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:31:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:31:25.546610 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b623ed10-a956-4830-af2b-7193ceabc123 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:25.564640 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b623ed10-a956-4830-af2b-7193ceabc123 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:25.675959 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fdc625-d3dd-4d7c-96f5-cf17bc48dd1a admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:25.684298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a95b58-e221-4ce8-b73f-f0a3ddce633e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:25.685040 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a95b58-e221-4ce8-b73f-f0a3ddce633e admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:25.685338 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a95b58-e221-4ce8-b73f-f0a3ddce633e admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:25.685619 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a95b58-e221-4ce8-b73f-f0a3ddce633e admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:25.686438 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a95b58-e221-4ce8-b73f-f0a3ddce633e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:25.688401 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a95b58-e221-4ce8-b73f-f0a3ddce633e admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:25.689085 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda0a3c-8919-49d3-9a57-d35b9a13e694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:25.689885 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda0a3c-8919-49d3-9a57-d35b9a13e694 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:25.690209 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda0a3c-8919-49d3-9a57-d35b9a13e694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:25.690501 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda0a3c-8919-49d3-9a57-d35b9a13e694 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:25.691205 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda0a3c-8919-49d3-9a57-d35b9a13e694 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:25.692850 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda0a3c-8919-49d3-9a57-d35b9a13e694 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:25.705207 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 908/1833] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:31:25.706529 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 926/1834] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:31:25.707245 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a95b58-e221-4ce8-b73f-f0a3ddce633e admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:25.707599 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda0a3c-8919-49d3-9a57-d35b9a13e694 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:25.721782 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda0a3c-8919-49d3-9a57-d35b9a13e694 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:25.723372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a95b58-e221-4ce8-b73f-f0a3ddce633e admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:25.724195 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:25.727482 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9281b4c-b43e-45f8-b49e-b0c5d850da17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:25.728262 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9281b4c-b43e-45f8-b49e-b0c5d850da17 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:25.728560 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9281b4c-b43e-45f8-b49e-b0c5d850da17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:25.728904 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9281b4c-b43e-45f8-b49e-b0c5d850da17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:25.729874 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9281b4c-b43e-45f8-b49e-b0c5d850da17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:25.731559 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5239ab5c-c811-4611-96a9-e3595c0c758a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:25.731832 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9281b4c-b43e-45f8-b49e-b0c5d850da17 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:25.743272 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239ab5c-c811-4611-96a9-e3595c0c758a admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:25.743670 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239ab5c-c811-4611-96a9-e3595c0c758a admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:25.743960 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239ab5c-c811-4611-96a9-e3595c0c758a admin admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:25.744715 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5239ab5c-c811-4611-96a9-e3595c0c758a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:25.748365 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 927/1835] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:31:25.749606 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9281b4c-b43e-45f8-b49e-b0c5d850da17 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:25.755651 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5239ab5c-c811-4611-96a9-e3595c0c758a admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:25.777055 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9281b4c-b43e-45f8-b49e-b0c5d850da17 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:25.777055 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f0f56-783e-47c5-8a00-ddf01caf20cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:25.777055 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f0f56-783e-47c5-8a00-ddf01caf20cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:25.777055 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f0f56-783e-47c5-8a00-ddf01caf20cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:25.777582 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f0f56-783e-47c5-8a00-ddf01caf20cd admin admin] PATH_INFO: `/v3/projects/87ce220cb82e438183b3483fc3917544/users/7a6efb71b0e74aaa86c6fb4993970bf8/roles/7c88c815db854954b516ab97cd471647` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:25.778666 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969f0f56-783e-47c5-8a00-ddf01caf20cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=87ce220cb82e438183b3483fc3917544, user_id=7a6efb71b0e74aaa86c6fb4993970bf8, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:25.788425 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969f0f56-783e-47c5-8a00-ddf01caf20cd admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:25.826664 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 928/1836] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:31:25 2022] PUT /identity/v3/projects/87ce220cb82e438183b3483fc3917544/users/7a6efb71b0e74aaa86c6fb4993970bf8/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:25.828028 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f0f56-783e-47c5-8a00-ddf01caf20cd admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:25.912448 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 909/1837] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:31:25 2022] POST /identity/v3/projects => generated 365 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:31:25.914049 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5239ab5c-c811-4611-96a9-e3595c0c758a admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:25.927228 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-5239ab5c-c811-4611-96a9-e3595c0c758a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:25.961014 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f0f56-783e-47c5-8a00-ddf01caf20cd admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:26.135954 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b706f61-af90-4327-a2df-453b0db1dd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:26.137482 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b706f61-af90-4327-a2df-453b0db1dd2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:26.137776 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b706f61-af90-4327-a2df-453b0db1dd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:26.137885 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5239ab5c-c811-4611-96a9-e3595c0c758a admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:26.141443 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b706f61-af90-4327-a2df-453b0db1dd2c admin admin] PATH_INFO: `/v3/projects/fac601f1d2954638a3a70e137048a69a/users/0607e0b0b98843da830fc85e100fcf2a/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:26.144951 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14456cad-60b4-4427-b5b7-7c216fa6e185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:26.146277 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14456cad-60b4-4427-b5b7-7c216fa6e185 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:26.146582 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14456cad-60b4-4427-b5b7-7c216fa6e185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:26.146912 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14456cad-60b4-4427-b5b7-7c216fa6e185 admin admin] PATH_INFO: `/v3/users/3181876ee88c4d218312bc3d7cc44502` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:26.148585 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14456cad-60b4-4427-b5b7-7c216fa6e185 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3181876ee88c4d218312bc3d7cc44502)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:26.151854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14456cad-60b4-4427-b5b7-7c216fa6e185 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:26.195364 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b706f61-af90-4327-a2df-453b0db1dd2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fac601f1d2954638a3a70e137048a69a, user_id=0607e0b0b98843da830fc85e100fcf2a, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:26.198607 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b706f61-af90-4327-a2df-453b0db1dd2c admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:26.456899 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 929/1838] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:31:25 2022] PUT /identity/v3/projects/fac601f1d2954638a3a70e137048a69a/users/0607e0b0b98843da830fc85e100fcf2a/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:26.458643 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034af43-0f0e-44dc-a3a2-7d3d52eabce6 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:26.458935 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034af43-0f0e-44dc-a3a2-7d3d52eabce6 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:26.459216 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034af43-0f0e-44dc-a3a2-7d3d52eabce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:26.493923 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:26.495483 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-9034af43-0f0e-44dc-a3a2-7d3d52eabce6 None None] MFA Rules not processed for user `7a6efb71b0e74aaa86c6fb4993970bf8`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:26.496997 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-9034af43-0f0e-44dc-a3a2-7d3d52eabce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:26.576820 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 930/1839] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:26 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:26.580113 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034af43-0f0e-44dc-a3a2-7d3d52eabce6 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:26.619649 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-14456cad-60b4-4427-b5b7-7c216fa6e185 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3181876ee88c4d218312bc3d7cc44502'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:26.638339 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-14456cad-60b4-4427-b5b7-7c216fa6e185 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3181876ee88c4d218312bc3d7cc44502'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:26.639119 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-14456cad-60b4-4427-b5b7-7c216fa6e185 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3181876ee88c4d218312bc3d7cc44502'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:26.653900 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-14456cad-60b4-4427-b5b7-7c216fa6e185 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3181876ee88c4d218312bc3d7cc44502'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:26.655031 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-14456cad-60b4-4427-b5b7-7c216fa6e185 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3181876ee88c4d218312bc3d7cc44502'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:26.660120 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034af43-0f0e-44dc-a3a2-7d3d52eabce6 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:26.665251 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9733cf29-35c0-4b86-b0f6-a5b5b26446a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:26.666012 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733cf29-35c0-4b86-b0f6-a5b5b26446a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:26.666321 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733cf29-35c0-4b86-b0f6-a5b5b26446a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:26.666713 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733cf29-35c0-4b86-b0f6-a5b5b26446a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:26.667393 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9733cf29-35c0-4b86-b0f6-a5b5b26446a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:26.669371 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9733cf29-35c0-4b86-b0f6-a5b5b26446a7 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:26.776682 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 931/1840] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:31:26 2022] POST /identity/v3/users => generated 283 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:31:26.789298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 910/1841] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:31:25 2022] DELETE /identity/v3/users/3181876ee88c4d218312bc3d7cc44502 => generated 0 bytes in 864 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:26.789298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14456cad-60b4-4427-b5b7-7c216fa6e185 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:26.790101 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c321fa0-1b91-48b5-aaab-450fe4bf3b71 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:26.790101 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c321fa0-1b91-48b5-aaab-450fe4bf3b71 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:26.790101 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c321fa0-1b91-48b5-aaab-450fe4bf3b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:26.790353 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-14456cad-60b4-4427-b5b7-7c216fa6e185 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:26.812664 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-0c321fa0-1b91-48b5-aaab-450fe4bf3b71 None None] MFA Rules not processed for user `0607e0b0b98843da830fc85e100fcf2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:26.813679 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-0c321fa0-1b91-48b5-aaab-450fe4bf3b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:26.892642 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 932/1842] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:26.893740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c321fa0-1b91-48b5-aaab-450fe4bf3b71 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:26.895189 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-0c321fa0-1b91-48b5-aaab-450fe4bf3b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:26.925360 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14456cad-60b4-4427-b5b7-7c216fa6e185 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:26.932187 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013cf529-aabe-495b-bde2-390b95b21544 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:26.933083 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013cf529-aabe-495b-bde2-390b95b21544 service cinder] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:26.933740 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013cf529-aabe-495b-bde2-390b95b21544 service cinder] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:26.934438 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013cf529-aabe-495b-bde2-390b95b21544 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:26.936579 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-013cf529-aabe-495b-bde2-390b95b21544 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:26.979096 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c321fa0-1b91-48b5-aaab-450fe4bf3b71 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:26.991150 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63dbeef-4461-44d0-9e65-261542610435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:26.992226 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63dbeef-4461-44d0-9e65-261542610435 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:26.992513 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63dbeef-4461-44d0-9e65-261542610435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:26.992794 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63dbeef-4461-44d0-9e65-261542610435 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:26.993684 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63dbeef-4461-44d0-9e65-261542610435 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:26.995851 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63dbeef-4461-44d0-9e65-261542610435 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:27.013745 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 933/1843] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:31:27.015469 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de672365-163e-4480-977e-660b8efca228 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:27.015716 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de672365-163e-4480-977e-660b8efca228 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:27.016011 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de672365-163e-4480-977e-660b8efca228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:27.056814 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-de672365-163e-4480-977e-660b8efca228 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:27.057847 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013cf529-aabe-495b-bde2-390b95b21544 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:27.058820 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013cf529-aabe-495b-bde2-390b95b21544 service cinder] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:27.059481 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-de672365-163e-4480-977e-660b8efca228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:27.071486 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 934/1844] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:27.072671 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de672365-163e-4480-977e-660b8efca228 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:27.074204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-de672365-163e-4480-977e-660b8efca228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:27.076909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 911/1845] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:31:26 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:31:27.078089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013cf529-aabe-495b-bde2-390b95b21544 service cinder] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:27.090724 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013cf529-aabe-495b-bde2-390b95b21544 service cinder] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:27.093596 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de672365-163e-4480-977e-660b8efca228 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:27.097085 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce962455-0bb9-4ec2-bc0d-8ac6f2321e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:27.097856 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce962455-0bb9-4ec2-bc0d-8ac6f2321e1d admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:27.098236 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce962455-0bb9-4ec2-bc0d-8ac6f2321e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:27.098612 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce962455-0bb9-4ec2-bc0d-8ac6f2321e1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:27.098916 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c52fad-b2f7-4b29-89cb-a2486133ac31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:27.099498 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce962455-0bb9-4ec2-bc0d-8ac6f2321e1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:27.099928 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c52fad-b2f7-4b29-89cb-a2486133ac31 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:27.100381 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c52fad-b2f7-4b29-89cb-a2486133ac31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:27.100819 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c52fad-b2f7-4b29-89cb-a2486133ac31 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:27.102142 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce962455-0bb9-4ec2-bc0d-8ac6f2321e1d admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:27.102635 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c52fad-b2f7-4b29-89cb-a2486133ac31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:27.104822 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c52fad-b2f7-4b29-89cb-a2486133ac31 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:27.117150 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 912/1846] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:31:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:31:27.117150 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce962455-0bb9-4ec2-bc0d-8ac6f2321e1d admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:27.118000 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-ce962455-0bb9-4ec2-bc0d-8ac6f2321e1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:27.181999 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce962455-0bb9-4ec2-bc0d-8ac6f2321e1d admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:27.185684 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b00b51-4a95-4706-bcb7-ae6b908b4f7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:27.186434 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b00b51-4a95-4706-bcb7-ae6b908b4f7d service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:27.186716 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b00b51-4a95-4706-bcb7-ae6b908b4f7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:27.187000 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b00b51-4a95-4706-bcb7-ae6b908b4f7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:27.197623 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b00b51-4a95-4706-bcb7-ae6b908b4f7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:27.199057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b00b51-4a95-4706-bcb7-ae6b908b4f7d service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:27.209218 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 913/1847] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:31:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:31:27.214452 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b00b51-4a95-4706-bcb7-ae6b908b4f7d service neutron] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:27.219840 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b00b51-4a95-4706-bcb7-ae6b908b4f7d service neutron] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:27.224869 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f528cbec-7289-47f8-b256-9679fd43d621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:27.225550 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528cbec-7289-47f8-b256-9679fd43d621 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:27.226385 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528cbec-7289-47f8-b256-9679fd43d621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:27.226385 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528cbec-7289-47f8-b256-9679fd43d621 admin admin] PATH_INFO: `/v3/projects/eef07a3eee044b9aa54fc568cb66cc0f/users/2c81dc9230574ce88c7828917e7bb660/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:27.244226 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f528cbec-7289-47f8-b256-9679fd43d621 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eef07a3eee044b9aa54fc568cb66cc0f, user_id=2c81dc9230574ce88c7828917e7bb660, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:27.245891 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 935/1848] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:31:27 2022] POST /identity/v3/projects => generated 373 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:31:27.246797 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c52fad-b2f7-4b29-89cb-a2486133ac31 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:27.246925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f528cbec-7289-47f8-b256-9679fd43d621 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:27.310649 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c52fad-b2f7-4b29-89cb-a2486133ac31 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:27.315973 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a63a29-7d46-4c98-8815-d660ba70d70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:27.316662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a63a29-7d46-4c98-8815-d660ba70d70e admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:27.316929 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a63a29-7d46-4c98-8815-d660ba70d70e admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:27.317211 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a63a29-7d46-4c98-8815-d660ba70d70e admin admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:27.317880 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a63a29-7d46-4c98-8815-d660ba70d70e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:27.319951 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a63a29-7d46-4c98-8815-d660ba70d70e admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:27.363991 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 914/1849] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:31:27 2022] PUT /identity/v3/projects/eef07a3eee044b9aa54fc568cb66cc0f/users/2c81dc9230574ce88c7828917e7bb660/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:27.375449 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db6894a-88c8-464d-bc7e-31627f45849a None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:27.376012 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db6894a-88c8-464d-bc7e-31627f45849a None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:27.377196 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db6894a-88c8-464d-bc7e-31627f45849a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:27.415391 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-9db6894a-88c8-464d-bc7e-31627f45849a None None] MFA Rules not processed for user `2c81dc9230574ce88c7828917e7bb660`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:27.417232 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-9db6894a-88c8-464d-bc7e-31627f45849a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:27.422698 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:27.463726 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 936/1850] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:31:27 2022] POST /identity/v3/users => generated 293 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:31:27.464204 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:27.465259 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a63a29-7d46-4c98-8815-d660ba70d70e admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:27.523865 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 915/1851] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:27.525053 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db6894a-88c8-464d-bc7e-31627f45849a None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:27.546946 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a63a29-7d46-4c98-8815-d660ba70d70e admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:27.668558 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09302e41-ea37-40ab-b51c-e26cc1e04d3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:27.669210 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09302e41-ea37-40ab-b51c-e26cc1e04d3f service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:27.669495 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09302e41-ea37-40ab-b51c-e26cc1e04d3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:27.669774 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09302e41-ea37-40ab-b51c-e26cc1e04d3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:27.672006 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-09302e41-ea37-40ab-b51c-e26cc1e04d3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:27.683496 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db6894a-88c8-464d-bc7e-31627f45849a None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:27.692322 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e47a1-9e97-43c3-ba3e-c47edabcae34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:27.693411 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e47a1-9e97-43c3-ba3e-c47edabcae34 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:27.693794 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e47a1-9e97-43c3-ba3e-c47edabcae34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:27.694203 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e47a1-9e97-43c3-ba3e-c47edabcae34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:27.695805 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e47a1-9e97-43c3-ba3e-c47edabcae34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:27.698928 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e47a1-9e97-43c3-ba3e-c47edabcae34 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:27.719347 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 916/1852] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:31:27 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:31:27.719347 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e47a1-9e97-43c3-ba3e-c47edabcae34 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:27.729672 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e47a1-9e97-43c3-ba3e-c47edabcae34 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:27.737063 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a5335-21b0-429a-83dd-f25b5f5a8e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:27.737752 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a5335-21b0-429a-83dd-f25b5f5a8e0e admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:27.738279 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a5335-21b0-429a-83dd-f25b5f5a8e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:27.738723 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a5335-21b0-429a-83dd-f25b5f5a8e0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:27.739548 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a5335-21b0-429a-83dd-f25b5f5a8e0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:27.739890 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:27.742604 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a5335-21b0-429a-83dd-f25b5f5a8e0e admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:27.762951 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09302e41-ea37-40ab-b51c-e26cc1e04d3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:27.764860 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09302e41-ea37-40ab-b51c-e26cc1e04d3f service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:27.777274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 937/1853] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:31:27 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:31:27.778152 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09302e41-ea37-40ab-b51c-e26cc1e04d3f service cinder] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:27.818753 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09302e41-ea37-40ab-b51c-e26cc1e04d3f service cinder] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:27.824342 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db5f993-d20d-4255-8dbf-1f7fcbf31912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:27.824674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5f993-d20d-4255-8dbf-1f7fcbf31912 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:27.824965 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5f993-d20d-4255-8dbf-1f7fcbf31912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:27.825250 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5f993-d20d-4255-8dbf-1f7fcbf31912 admin admin] PATH_INFO: `/v3/projects/90ecfd292fc9456cacd8db92f8fc7a97/users/b463e71faa6b4c889b8cbbaecdac42c1/roles/7c88c815db854954b516ab97cd471647` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:27.840220 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 917/1854] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:31:27 2022] POST /identity/v3/projects => generated 367 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:31:27.843160 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a5335-21b0-429a-83dd-f25b5f5a8e0e admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:27.854556 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db5f993-d20d-4255-8dbf-1f7fcbf31912 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90ecfd292fc9456cacd8db92f8fc7a97, user_id=b463e71faa6b4c889b8cbbaecdac42c1, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:27.858132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db5f993-d20d-4255-8dbf-1f7fcbf31912 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:27.917532 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 938/1855] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:31:27 2022] PUT /identity/v3/projects/90ecfd292fc9456cacd8db92f8fc7a97/users/b463e71faa6b4c889b8cbbaecdac42c1/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:27.918918 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db5f993-d20d-4255-8dbf-1f7fcbf31912 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:27.920531 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a5335-21b0-429a-83dd-f25b5f5a8e0e admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:27.979155 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2593047-1ada-4b04-af62-c7a296acb415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:27.980161 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2593047-1ada-4b04-af62-c7a296acb415 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:27.980570 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2593047-1ada-4b04-af62-c7a296acb415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:27.980943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2593047-1ada-4b04-af62-c7a296acb415 admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:27.981824 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2593047-1ada-4b04-af62-c7a296acb415 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:27.984788 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2593047-1ada-4b04-af62-c7a296acb415 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:27.987543 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db5f993-d20d-4255-8dbf-1f7fcbf31912 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:27.992820 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6abf614-0d4c-499b-9373-45b9c29c045a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:27.993543 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6abf614-0d4c-499b-9373-45b9c29c045a admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:27.993900 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6abf614-0d4c-499b-9373-45b9c29c045a admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:27.994254 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6abf614-0d4c-499b-9373-45b9c29c045a admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:27.995219 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6abf614-0d4c-499b-9373-45b9c29c045a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:27.996903 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6abf614-0d4c-499b-9373-45b9c29c045a admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:28.012812 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 939/1856] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:31:27 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:31:28.012919 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6abf614-0d4c-499b-9373-45b9c29c045a admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:28.025330 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6abf614-0d4c-499b-9373-45b9c29c045a admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:28.030319 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2907d5-921f-4b14-b613-1f808eec6278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:28.033394 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2907d5-921f-4b14-b613-1f808eec6278 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:28.033875 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2907d5-921f-4b14-b613-1f808eec6278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:28.034182 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2907d5-921f-4b14-b613-1f808eec6278 admin admin] PATH_INFO: `/v3/projects/90ecfd292fc9456cacd8db92f8fc7a97/users/b463e71faa6b4c889b8cbbaecdac42c1/roles/7c88c815db854954b516ab97cd471647` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:28.036749 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2907d5-921f-4b14-b613-1f808eec6278 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90ecfd292fc9456cacd8db92f8fc7a97, user_id=b463e71faa6b4c889b8cbbaecdac42c1, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:28.039441 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2907d5-921f-4b14-b613-1f808eec6278 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:28.051718 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 940/1857] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:31:28 2022] PUT /identity/v3/projects/90ecfd292fc9456cacd8db92f8fc7a97/users/b463e71faa6b4c889b8cbbaecdac42c1/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:28.061698 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2907d5-921f-4b14-b613-1f808eec6278 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:28.063534 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-5f2907d5-921f-4b14-b613-1f808eec6278 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:28.097865 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 918/1858] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:31:27 2022] POST /identity/v3/users => generated 290 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:31:28.101901 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f792ed2-7e9c-444e-b5dc-acfa31997796 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:28.101901 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f792ed2-7e9c-444e-b5dc-acfa31997796 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:28.101901 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f792ed2-7e9c-444e-b5dc-acfa31997796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:28.119514 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:28.134827 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-4f792ed2-7e9c-444e-b5dc-acfa31997796 None None] MFA Rules not processed for user `b463e71faa6b4c889b8cbbaecdac42c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:28.136796 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-4f792ed2-7e9c-444e-b5dc-acfa31997796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:28.143105 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2907d5-921f-4b14-b613-1f808eec6278 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:28.152194 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9694e983-6209-4ae8-a213-da144f05bb48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:28.152849 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694e983-6209-4ae8-a213-da144f05bb48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:28.153138 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694e983-6209-4ae8-a213-da144f05bb48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:28.153413 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694e983-6209-4ae8-a213-da144f05bb48 admin admin] PATH_INFO: `/v3/projects/413e184357684d2db032404bfeeb098e` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:28.154599 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9694e983-6209-4ae8-a213-da144f05bb48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=413e184357684d2db032404bfeeb098e)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:28.156580 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9694e983-6209-4ae8-a213-da144f05bb48 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:28.214230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 919/1859] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:28 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:28.215241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f792ed2-7e9c-444e-b5dc-acfa31997796 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:28.228897 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f792ed2-7e9c-444e-b5dc-acfa31997796 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:28.233853 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544a43a8-e056-4968-84fc-8d80215fc9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:28.234522 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a43a8-e056-4968-84fc-8d80215fc9f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:28.234812 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a43a8-e056-4968-84fc-8d80215fc9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:28.235126 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a43a8-e056-4968-84fc-8d80215fc9f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:28.235858 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544a43a8-e056-4968-84fc-8d80215fc9f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:28.237553 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544a43a8-e056-4968-84fc-8d80215fc9f6 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:28.250665 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 920/1860] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:31:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:31:28.250665 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544a43a8-e056-4968-84fc-8d80215fc9f6 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:28.253871 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-544a43a8-e056-4968-84fc-8d80215fc9f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:28.258542 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:28.281819 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-9694e983-6209-4ae8-a213-da144f05bb48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '413e184357684d2db032404bfeeb098e', 'request_id': 'req-9694e983-6209-4ae8-a213-da144f05bb48'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:28.289727 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-9694e983-6209-4ae8-a213-da144f05bb48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '413e184357684d2db032404bfeeb098e', 'request_id': 'req-9694e983-6209-4ae8-a213-da144f05bb48'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:28.295422 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-9694e983-6209-4ae8-a213-da144f05bb48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '413e184357684d2db032404bfeeb098e', 'request_id': 'req-9694e983-6209-4ae8-a213-da144f05bb48'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:28.332289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544a43a8-e056-4968-84fc-8d80215fc9f6 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:28.381919 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a9d36-8083-4cb3-8b33-4ef00cdc8c1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:28.382155 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-9694e983-6209-4ae8-a213-da144f05bb48 admin admin] The token cache is being invalidate because project 413e184357684d2db032404bfeeb098e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:31:28.382259 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-9694e983-6209-4ae8-a213-da144f05bb48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:28.382491 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a9d36-8083-4cb3-8b33-4ef00cdc8c1e service cinder] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:28.382810 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a9d36-8083-4cb3-8b33-4ef00cdc8c1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:28.382917 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-9694e983-6209-4ae8-a213-da144f05bb48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:28.383094 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a9d36-8083-4cb3-8b33-4ef00cdc8c1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:28.384525 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 941/1861] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:31:28 2022] DELETE /identity/v3/projects/413e184357684d2db032404bfeeb098e => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:28.385227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-880a9d36-8083-4cb3-8b33-4ef00cdc8c1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:28.385343 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9694e983-6209-4ae8-a213-da144f05bb48 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:28.386519 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-9694e983-6209-4ae8-a213-da144f05bb48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:28.448796 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880a9d36-8083-4cb3-8b33-4ef00cdc8c1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:28.449702 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9694e983-6209-4ae8-a213-da144f05bb48 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:28.450243 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880a9d36-8083-4cb3-8b33-4ef00cdc8c1e service cinder] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:28.455072 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5491d9-e2fd-4c87-8574-20fbd3c97a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:28.455751 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5491d9-e2fd-4c87-8574-20fbd3c97a12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:28.461692 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5491d9-e2fd-4c87-8574-20fbd3c97a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:28.461692 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5491d9-e2fd-4c87-8574-20fbd3c97a12 admin admin] PATH_INFO: `/v3/projects/fcb729706f6c4f908886cd7271ce808d/users/0b25f0dc047143da963336ed1fa16f6d/roles/7c88c815db854954b516ab97cd471647` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:28.461902 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 921/1862] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:31:28 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:31:28.461902 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a9d36-8083-4cb3-8b33-4ef00cdc8c1e service cinder] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:28.462840 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-880a9d36-8083-4cb3-8b33-4ef00cdc8c1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:28.471626 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5491d9-e2fd-4c87-8574-20fbd3c97a12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcb729706f6c4f908886cd7271ce808d, user_id=0b25f0dc047143da963336ed1fa16f6d, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:28.475430 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5491d9-e2fd-4c87-8574-20fbd3c97a12 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:28.476278 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a9d36-8083-4cb3-8b33-4ef00cdc8c1e service cinder] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:28.481055 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd2fa7-7585-4fbe-ac6b-89dc2606f06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:28.481698 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd2fa7-7585-4fbe-ac6b-89dc2606f06e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:28.481983 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd2fa7-7585-4fbe-ac6b-89dc2606f06e admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:28.482260 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd2fa7-7585-4fbe-ac6b-89dc2606f06e admin admin] PATH_INFO: `/v3/users/ff6dee211eb642579972ee81767402f0` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:28.483439 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cd2fa7-7585-4fbe-ac6b-89dc2606f06e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff6dee211eb642579972ee81767402f0)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:28.485443 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cd2fa7-7585-4fbe-ac6b-89dc2606f06e admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:28.572845 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 942/1863] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:31:28 2022] PUT /identity/v3/projects/fcb729706f6c4f908886cd7271ce808d/users/0b25f0dc047143da963336ed1fa16f6d/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:28.581888 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5491d9-e2fd-4c87-8574-20fbd3c97a12 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:28.646718 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5491d9-e2fd-4c87-8574-20fbd3c97a12 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:28.651710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73898616-8d64-42f3-b6c5-c9cb3186dde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:28.652367 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73898616-8d64-42f3-b6c5-c9cb3186dde8 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:28.652645 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73898616-8d64-42f3-b6c5-c9cb3186dde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:28.652924 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73898616-8d64-42f3-b6c5-c9cb3186dde8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:28.653835 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73898616-8d64-42f3-b6c5-c9cb3186dde8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:28.655500 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73898616-8d64-42f3-b6c5-c9cb3186dde8 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:28.666224 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 943/1864] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:31:28 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 15:31:28.670404 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73898616-8d64-42f3-b6c5-c9cb3186dde8 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:28.672149 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-73898616-8d64-42f3-b6c5-c9cb3186dde8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:28.731879 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73898616-8d64-42f3-b6c5-c9cb3186dde8 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:28.732537 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-22cd2fa7-7585-4fbe-ac6b-89dc2606f06e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff6dee211eb642579972ee81767402f0'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:28.736243 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a718d3c0-8caa-435f-9af0-b3f2b3ea2a0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:28.736870 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718d3c0-8caa-435f-9af0-b3f2b3ea2a0f service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:28.737161 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718d3c0-8caa-435f-9af0-b3f2b3ea2a0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:28.737554 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718d3c0-8caa-435f-9af0-b3f2b3ea2a0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:28.739239 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-a718d3c0-8caa-435f-9af0-b3f2b3ea2a0f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:28.740526 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-22cd2fa7-7585-4fbe-ac6b-89dc2606f06e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff6dee211eb642579972ee81767402f0'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:28.741241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-22cd2fa7-7585-4fbe-ac6b-89dc2606f06e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff6dee211eb642579972ee81767402f0'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:28.754167 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-22cd2fa7-7585-4fbe-ac6b-89dc2606f06e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff6dee211eb642579972ee81767402f0'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:28.755932 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-22cd2fa7-7585-4fbe-ac6b-89dc2606f06e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff6dee211eb642579972ee81767402f0'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:28.800310 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a718d3c0-8caa-435f-9af0-b3f2b3ea2a0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:28.801760 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a718d3c0-8caa-435f-9af0-b3f2b3ea2a0f service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:28.810760 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 944/1865] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:31:28 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 15:31:28.813300 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a718d3c0-8caa-435f-9af0-b3f2b3ea2a0f service cinder] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:28.823124 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a718d3c0-8caa-435f-9af0-b3f2b3ea2a0f service cinder] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:28.827782 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f35f79-9d2e-4a49-bca4-1416dd6dfd7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:28.828506 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f35f79-9d2e-4a49-bca4-1416dd6dfd7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:28.828797 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f35f79-9d2e-4a49-bca4-1416dd6dfd7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:28.829076 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f35f79-9d2e-4a49-bca4-1416dd6dfd7d admin admin] PATH_INFO: `/v3/projects/fcb729706f6c4f908886cd7271ce808d/users/0b25f0dc047143da963336ed1fa16f6d/roles/7c88c815db854954b516ab97cd471647` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:28.831822 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f35f79-9d2e-4a49-bca4-1416dd6dfd7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcb729706f6c4f908886cd7271ce808d, user_id=0b25f0dc047143da963336ed1fa16f6d, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:28.834790 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f35f79-9d2e-4a49-bca4-1416dd6dfd7d admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:28.847239 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 945/1866] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:31:28 2022] PUT /identity/v3/projects/fcb729706f6c4f908886cd7271ce808d/users/0b25f0dc047143da963336ed1fa16f6d/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:28.858796 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb4b65-ca7e-4533-98bc-52b3dc850b97 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:28.859081 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb4b65-ca7e-4533-98bc-52b3dc850b97 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:28.859369 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb4b65-ca7e-4533-98bc-52b3dc850b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:28.868552 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 922/1867] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:31:28 2022] DELETE /identity/v3/users/ff6dee211eb642579972ee81767402f0 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:28.906658 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-cfcb4b65-ca7e-4533-98bc-52b3dc850b97 None None] MFA Rules not processed for user `0b25f0dc047143da963336ed1fa16f6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:28.907922 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-cfcb4b65-ca7e-4533-98bc-52b3dc850b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:28.991351 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 946/1868] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:28 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:29.004689 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd2fa7-7585-4fbe-ac6b-89dc2606f06e admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:29.006632 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-22cd2fa7-7585-4fbe-ac6b-89dc2606f06e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:29.082914 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd2fa7-7585-4fbe-ac6b-89dc2606f06e admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:29.086814 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892927fb-1693-4628-b562-c09ec9a4ce3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:29.087454 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892927fb-1693-4628-b562-c09ec9a4ce3a service cinder] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:29.087883 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892927fb-1693-4628-b562-c09ec9a4ce3a service cinder] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:29.088338 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892927fb-1693-4628-b562-c09ec9a4ce3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:29.112818 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892927fb-1693-4628-b562-c09ec9a4ce3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:29.114280 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892927fb-1693-4628-b562-c09ec9a4ce3a service cinder] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:29.125013 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 923/1869] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:31:29 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:31:29.251840 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:29.424641 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcb4b65-ca7e-4533-98bc-52b3dc850b97 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:29.426636 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-cfcb4b65-ca7e-4533-98bc-52b3dc850b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:29.444006 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcb4b65-ca7e-4533-98bc-52b3dc850b97 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:29.448922 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e59d79e-d629-48c2-9140-1dcf728f7eb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:29.449952 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59d79e-d629-48c2-9140-1dcf728f7eb0 service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:29.450161 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59d79e-d629-48c2-9140-1dcf728f7eb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:29.450579 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59d79e-d629-48c2-9140-1dcf728f7eb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:29.452720 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-7e59d79e-d629-48c2-9140-1dcf728f7eb0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:29.527626 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:29.563407 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e59d79e-d629-48c2-9140-1dcf728f7eb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:29.564945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e59d79e-d629-48c2-9140-1dcf728f7eb0 service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:29.574697 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 947/1870] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:31:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:31:29.619818 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892927fb-1693-4628-b562-c09ec9a4ce3a service cinder] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:29.621488 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-892927fb-1693-4628-b562-c09ec9a4ce3a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:29.761033 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892927fb-1693-4628-b562-c09ec9a4ce3a service cinder] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:29.767574 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c337a167-a2ba-4f16-b85c-c662de5c32e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:29.768438 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337a167-a2ba-4f16-b85c-c662de5c32e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:29.768897 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337a167-a2ba-4f16-b85c-c662de5c32e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:29.769362 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337a167-a2ba-4f16-b85c-c662de5c32e6 admin admin] PATH_INFO: `/v3/projects/4f3b25ec2aa84c60b10ae9d34757f1bb` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:29.772043 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c337a167-a2ba-4f16-b85c-c662de5c32e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f3b25ec2aa84c60b10ae9d34757f1bb)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:29.775205 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c337a167-a2ba-4f16-b85c-c662de5c32e6 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:29.918006 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-c337a167-a2ba-4f16-b85c-c662de5c32e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f3b25ec2aa84c60b10ae9d34757f1bb', 'request_id': 'req-c337a167-a2ba-4f16-b85c-c662de5c32e6'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:29.929692 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-c337a167-a2ba-4f16-b85c-c662de5c32e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f3b25ec2aa84c60b10ae9d34757f1bb', 'request_id': 'req-c337a167-a2ba-4f16-b85c-c662de5c32e6'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:30.006447 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-c337a167-a2ba-4f16-b85c-c662de5c32e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f3b25ec2aa84c60b10ae9d34757f1bb', 'request_id': 'req-c337a167-a2ba-4f16-b85c-c662de5c32e6'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:30.021421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-c337a167-a2ba-4f16-b85c-c662de5c32e6 admin admin] The token cache is being invalidate because project 4f3b25ec2aa84c60b10ae9d34757f1bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:31:30.021720 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-c337a167-a2ba-4f16-b85c-c662de5c32e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:30.022400 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-c337a167-a2ba-4f16-b85c-c662de5c32e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:30.024462 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 924/1871] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:31:29 2022] DELETE /identity/v3/projects/4f3b25ec2aa84c60b10ae9d34757f1bb => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:30.029347 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8cdff-ad15-466c-b22f-277443955ccc None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:30.029848 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8cdff-ad15-466c-b22f-277443955ccc None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:30.030046 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8cdff-ad15-466c-b22f-277443955ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:30.052610 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-53e8cdff-ad15-466c-b22f-277443955ccc None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:30.053941 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-53e8cdff-ad15-466c-b22f-277443955ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:30.139101 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 948/1872] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:30.149474 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4ef1e-6f79-41f3-8341-90e5793011dd None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:30.149764 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4ef1e-6f79-41f3-8341-90e5793011dd None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:30.150042 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4ef1e-6f79-41f3-8341-90e5793011dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:30.175292 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-02c4ef1e-6f79-41f3-8341-90e5793011dd None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:30.176747 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-02c4ef1e-6f79-41f3-8341-90e5793011dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:30.202657 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 925/1873] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:31:30.205503 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e8cdff-ad15-466c-b22f-277443955ccc None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:30.214060 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:30.219130 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e8cdff-ad15-466c-b22f-277443955ccc None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:30.245581 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580818c6-6b01-4f28-a0dc-412b01d2febc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:30.246369 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580818c6-6b01-4f28-a0dc-412b01d2febc admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:30.246758 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580818c6-6b01-4f28-a0dc-412b01d2febc admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:30.247140 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580818c6-6b01-4f28-a0dc-412b01d2febc admin admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:30.247983 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580818c6-6b01-4f28-a0dc-412b01d2febc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:31:30.248420 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580818c6-6b01-4f28-a0dc-412b01d2febc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:30.250249 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580818c6-6b01-4f28-a0dc-412b01d2febc admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:30.266871 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 949/1874] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:31:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:31:30.277007 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c4ef1e-6f79-41f3-8341-90e5793011dd None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:30.290610 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c4ef1e-6f79-41f3-8341-90e5793011dd None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:30.299545 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806515f2-9543-48b0-9616-2617c52f2714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:30.299985 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806515f2-9543-48b0-9616-2617c52f2714 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:30.300402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806515f2-9543-48b0-9616-2617c52f2714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:30.300856 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806515f2-9543-48b0-9616-2617c52f2714 admin admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:30.303031 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806515f2-9543-48b0-9616-2617c52f2714 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:30.306146 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806515f2-9543-48b0-9616-2617c52f2714 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:30.430942 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 926/1875] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:31:30 2022] POST /identity/v3/projects => generated 381 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:31:30.439633 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580818c6-6b01-4f28-a0dc-412b01d2febc admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:30.456462 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580818c6-6b01-4f28-a0dc-412b01d2febc admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:30.549009 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c06086f-1177-4913-829f-e543e91acd98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:30.549725 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06086f-1177-4913-829f-e543e91acd98 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:30.550017 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06086f-1177-4913-829f-e543e91acd98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:30.550309 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06086f-1177-4913-829f-e543e91acd98 admin admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:30.550988 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c06086f-1177-4913-829f-e543e91acd98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:30.553325 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c06086f-1177-4913-829f-e543e91acd98 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:30.696806 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 950/1876] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:31:30 2022] POST /identity/v3/users => generated 291 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:31:30.700867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806515f2-9543-48b0-9616-2617c52f2714 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:30.712907 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806515f2-9543-48b0-9616-2617c52f2714 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:30.718272 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9bdef-6fb4-44aa-8656-00d317bb45cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:30.719002 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9bdef-6fb4-44aa-8656-00d317bb45cf admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:30.719479 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9bdef-6fb4-44aa-8656-00d317bb45cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:30.720243 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9bdef-6fb4-44aa-8656-00d317bb45cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:30.721168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce9bdef-6fb4-44aa-8656-00d317bb45cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:30.723220 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce9bdef-6fb4-44aa-8656-00d317bb45cf admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:30.748588 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 927/1877] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 15:31:30.753406 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c06086f-1177-4913-829f-e543e91acd98 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:30.764957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c06086f-1177-4913-829f-e543e91acd98 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:30.769705 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9e8f8-5846-4683-9333-541849429edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:30.770329 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9e8f8-5846-4683-9333-541849429edb admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:30.770619 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9e8f8-5846-4683-9333-541849429edb admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:30.770900 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9e8f8-5846-4683-9333-541849429edb admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:30.771617 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae9e8f8-5846-4683-9333-541849429edb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:30.773277 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae9e8f8-5846-4683-9333-541849429edb admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:30.784371 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 951/1878] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:31:30.788986 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9bdef-6fb4-44aa-8656-00d317bb45cf admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:30.802318 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9bdef-6fb4-44aa-8656-00d317bb45cf admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:30.808853 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2d059a-8de8-4241-8e62-13eec1c9ab78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:30.809730 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d059a-8de8-4241-8e62-13eec1c9ab78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:30.810115 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d059a-8de8-4241-8e62-13eec1c9ab78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:30.810574 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d059a-8de8-4241-8e62-13eec1c9ab78 admin admin] PATH_INFO: `/v3/projects/e531255ed991445f90e3285f48158313/users/43b5852d6657448fbd510f504448faea/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:30.829580 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2d059a-8de8-4241-8e62-13eec1c9ab78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e531255ed991445f90e3285f48158313, user_id=43b5852d6657448fbd510f504448faea, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:30.832537 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2d059a-8de8-4241-8e62-13eec1c9ab78 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:30.904184 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:30.914826 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 928/1879] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:31:30 2022] PUT /identity/v3/projects/e531255ed991445f90e3285f48158313/users/43b5852d6657448fbd510f504448faea/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:30.925844 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e04c9d-bcb6-4584-8a9c-14cb8b8842ff None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:30.926386 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e04c9d-bcb6-4584-8a9c-14cb8b8842ff None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:30.926842 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e04c9d-bcb6-4584-8a9c-14cb8b8842ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:30.977868 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-31e04c9d-bcb6-4584-8a9c-14cb8b8842ff None None] MFA Rules not processed for user `43b5852d6657448fbd510f504448faea`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:30.979178 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-31e04c9d-bcb6-4584-8a9c-14cb8b8842ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:31.085678 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 952/1880] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:31.091490 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2d059a-8de8-4241-8e62-13eec1c9ab78 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:31.211931 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2d059a-8de8-4241-8e62-13eec1c9ab78 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:31.217876 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745bbc6-d6d4-4946-987e-037b8be2a0fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:31.218704 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745bbc6-d6d4-4946-987e-037b8be2a0fa admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:31.219130 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745bbc6-d6d4-4946-987e-037b8be2a0fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:31.222108 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745bbc6-d6d4-4946-987e-037b8be2a0fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:31.222108 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9745bbc6-d6d4-4946-987e-037b8be2a0fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:31.223703 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9745bbc6-d6d4-4946-987e-037b8be2a0fa admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:31.310067 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 929/1881] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:31:31 2022] POST /identity/v3/projects => generated 381 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:31:31.312136 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e04c9d-bcb6-4584-8a9c-14cb8b8842ff None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:31.329063 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e04c9d-bcb6-4584-8a9c-14cb8b8842ff None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:31.444227 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aefc01-bb98-4a77-b4ee-866d466466b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:31.445026 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aefc01-bb98-4a77-b4ee-866d466466b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:31.445452 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aefc01-bb98-4a77-b4ee-866d466466b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:31.446839 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aefc01-bb98-4a77-b4ee-866d466466b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:31.447801 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aefc01-bb98-4a77-b4ee-866d466466b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:31.450733 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aefc01-bb98-4a77-b4ee-866d466466b6 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:31.573776 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 953/1882] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:31:31 2022] POST /identity/v3/users => generated 297 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:31:31.578665 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745bbc6-d6d4-4946-987e-037b8be2a0fa admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:31.599141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745bbc6-d6d4-4946-987e-037b8be2a0fa admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:31.611551 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc389fe-5e0c-402a-be33-40656fac45cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:31.612238 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc389fe-5e0c-402a-be33-40656fac45cc admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:31.612529 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc389fe-5e0c-402a-be33-40656fac45cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:31.612809 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc389fe-5e0c-402a-be33-40656fac45cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:31.613562 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc389fe-5e0c-402a-be33-40656fac45cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:31.618200 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc389fe-5e0c-402a-be33-40656fac45cc admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:31.655853 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 930/1883] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:31:31 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:31:31.660310 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aefc01-bb98-4a77-b4ee-866d466466b6 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:31.674437 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aefc01-bb98-4a77-b4ee-866d466466b6 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:31.697013 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f531269-1e0e-4a47-9b3f-e00a49a511bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:31.697640 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f531269-1e0e-4a47-9b3f-e00a49a511bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:31.698023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f531269-1e0e-4a47-9b3f-e00a49a511bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:31.698394 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f531269-1e0e-4a47-9b3f-e00a49a511bc admin admin] PATH_INFO: `/v3/projects/ad6b94324205456687f28eb3ff9c24c7/users/2e271ace6a07409086817e93573c49af/roles/7c88c815db854954b516ab97cd471647` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:31.722280 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f531269-1e0e-4a47-9b3f-e00a49a511bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad6b94324205456687f28eb3ff9c24c7, user_id=2e271ace6a07409086817e93573c49af, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:31.726843 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f531269-1e0e-4a47-9b3f-e00a49a511bc admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:31.847779 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f9b48-ae4a-4dcd-b7e5-6cfc034415f2 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:31.847779 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f9b48-ae4a-4dcd-b7e5-6cfc034415f2 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:31.847779 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f9b48-ae4a-4dcd-b7e5-6cfc034415f2 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:31.847779 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 931/1884] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 5 15:31:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 15:31:31.847779 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1fd550-8bfa-407c-99b3-807cfe2e0419 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:31.847779 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1fd550-8bfa-407c-99b3-807cfe2e0419 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:31.848626 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1fd550-8bfa-407c-99b3-807cfe2e0419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:31.864954 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-5b1fd550-8bfa-407c-99b3-807cfe2e0419 None None] MFA Rules not processed for user `bc0497dab1f44bb7b8196d39bf01d770`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:31.866624 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-5b1fd550-8bfa-407c-99b3-807cfe2e0419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:31.950922 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 932/1885] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 5 15:31:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:31:31.963740 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1fd550-8bfa-407c-99b3-807cfe2e0419 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:31.965159 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-5b1fd550-8bfa-407c-99b3-807cfe2e0419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:31.979316 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1fd550-8bfa-407c-99b3-807cfe2e0419 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:31.985293 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f996d-c8ac-41d6-a04f-565a6004530c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:31.985965 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f996d-c8ac-41d6-a04f-565a6004530c service nova] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:31.986203 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f996d-c8ac-41d6-a04f-565a6004530c service nova] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:31.986484 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f996d-c8ac-41d6-a04f-565a6004530c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:31.995758 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3f996d-c8ac-41d6-a04f-565a6004530c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:31.997376 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3f996d-c8ac-41d6-a04f-565a6004530c service nova] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:32.007927 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 933/1886] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:31:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 15:31:32.760527 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 954/1887] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:31:31 2022] PUT /identity/v3/projects/ad6b94324205456687f28eb3ff9c24c7/users/2e271ace6a07409086817e93573c49af/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 1097 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:32.761240 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f996d-c8ac-41d6-a04f-565a6004530c service nova] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:32.781916 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f996d-c8ac-41d6-a04f-565a6004530c service nova] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:32.876303 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d832d160-9368-4289-bfef-abe5891bb5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:32.878224 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832d160-9368-4289-bfef-abe5891bb5b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:32.878667 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832d160-9368-4289-bfef-abe5891bb5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:32.879179 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832d160-9368-4289-bfef-abe5891bb5b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:32.880056 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d832d160-9368-4289-bfef-abe5891bb5b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:32.881962 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d832d160-9368-4289-bfef-abe5891bb5b1 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:32.911532 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f531269-1e0e-4a47-9b3f-e00a49a511bc admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:32.914139 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 934/1888] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:31:32 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:31:32.924850 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f531269-1e0e-4a47-9b3f-e00a49a511bc admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:32.930415 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1cebce-4873-4ee4-a3c5-3d2a8853e0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:32.931446 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cebce-4873-4ee4-a3c5-3d2a8853e0e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:32.931920 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cebce-4873-4ee4-a3c5-3d2a8853e0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:32.932720 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cebce-4873-4ee4-a3c5-3d2a8853e0e7 admin admin] PATH_INFO: `/v3/projects/ad6b94324205456687f28eb3ff9c24c7/users/2e271ace6a07409086817e93573c49af/roles/7c88c815db854954b516ab97cd471647` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:32.935670 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1cebce-4873-4ee4-a3c5-3d2a8853e0e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad6b94324205456687f28eb3ff9c24c7, user_id=2e271ace6a07409086817e93573c49af, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:32.938703 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1cebce-4873-4ee4-a3c5-3d2a8853e0e7 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:32.965547 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 955/1889] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:31:32 2022] PUT /identity/v3/projects/ad6b94324205456687f28eb3ff9c24c7/users/2e271ace6a07409086817e93573c49af/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:31:32.966493 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5934d92-7351-47c1-b69e-13138b7b7262 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:32.967231 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5934d92-7351-47c1-b69e-13138b7b7262 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:32.967231 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5934d92-7351-47c1-b69e-13138b7b7262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:33.012425 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-c5934d92-7351-47c1-b69e-13138b7b7262 None None] MFA Rules not processed for user `2e271ace6a07409086817e93573c49af`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:33.014023 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-c5934d92-7351-47c1-b69e-13138b7b7262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:33.202588 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 935/1890] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:32 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:33.218728 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1cebce-4873-4ee4-a3c5-3d2a8853e0e7 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:33.221287 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-cf1cebce-4873-4ee4-a3c5-3d2a8853e0e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:33.307656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1cebce-4873-4ee4-a3c5-3d2a8853e0e7 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:33.313177 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2a5df-894a-4517-a5ad-9ba6b7fbb50b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:33.313814 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2a5df-894a-4517-a5ad-9ba6b7fbb50b service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:33.314746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2a5df-894a-4517-a5ad-9ba6b7fbb50b service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:33.315115 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2a5df-894a-4517-a5ad-9ba6b7fbb50b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:33.329455 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b2a5df-894a-4517-a5ad-9ba6b7fbb50b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:33.331504 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b2a5df-894a-4517-a5ad-9ba6b7fbb50b service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:33.344268 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 956/1891] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:31:33 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:31:33.902551 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:34.041208 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:34.590552 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5934d92-7351-47c1-b69e-13138b7b7262 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:34.599800 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5934d92-7351-47c1-b69e-13138b7b7262 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:34.607245 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1702d10-ad30-4283-83f5-c482ba0b3bc3 tempest-VolumeQuotaClassesTest-2102805714 tempest-VolumeQuotaClassesTest-2102805714-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e271ace6a07409086817e93573c49af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad6b94324205456687f28eb3ff9c24c7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:34.607874 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1702d10-ad30-4283-83f5-c482ba0b3bc3 tempest-VolumeQuotaClassesTest-2102805714 tempest-VolumeQuotaClassesTest-2102805714-project-admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:34.608353 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1702d10-ad30-4283-83f5-c482ba0b3bc3 tempest-VolumeQuotaClassesTest-2102805714 tempest-VolumeQuotaClassesTest-2102805714-project-admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:34.608792 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1702d10-ad30-4283-83f5-c482ba0b3bc3 tempest-VolumeQuotaClassesTest-2102805714 tempest-VolumeQuotaClassesTest-2102805714-project-admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:34.610320 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1702d10-ad30-4283-83f5-c482ba0b3bc3 tempest-VolumeQuotaClassesTest-2102805714 tempest-VolumeQuotaClassesTest-2102805714-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:31:34.610815 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1702d10-ad30-4283-83f5-c482ba0b3bc3 tempest-VolumeQuotaClassesTest-2102805714 tempest-VolumeQuotaClassesTest-2102805714-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:34.613400 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1702d10-ad30-4283-83f5-c482ba0b3bc3 tempest-VolumeQuotaClassesTest-2102805714 tempest-VolumeQuotaClassesTest-2102805714-project-admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:34.648101 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 936/1892] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:31:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:31:34.652255 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2a5df-894a-4517-a5ad-9ba6b7fbb50b service cinder] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:34.662247 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2a5df-894a-4517-a5ad-9ba6b7fbb50b service cinder] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:34.677100 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65fba7-e827-4144-ad8b-085bb702ec76 tempest-VolumeQuotaClassesTest-2102805714 tempest-VolumeQuotaClassesTest-2102805714-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e271ace6a07409086817e93573c49af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad6b94324205456687f28eb3ff9c24c7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:34.677743 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65fba7-e827-4144-ad8b-085bb702ec76 tempest-VolumeQuotaClassesTest-2102805714 tempest-VolumeQuotaClassesTest-2102805714-project-admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:34.678039 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65fba7-e827-4144-ad8b-085bb702ec76 tempest-VolumeQuotaClassesTest-2102805714 tempest-VolumeQuotaClassesTest-2102805714-project-admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:34.678328 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65fba7-e827-4144-ad8b-085bb702ec76 tempest-VolumeQuotaClassesTest-2102805714 tempest-VolumeQuotaClassesTest-2102805714-project-admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:34.679027 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d65fba7-e827-4144-ad8b-085bb702ec76 tempest-VolumeQuotaClassesTest-2102805714 tempest-VolumeQuotaClassesTest-2102805714-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:34.680999 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d65fba7-e827-4144-ad8b-085bb702ec76 tempest-VolumeQuotaClassesTest-2102805714 tempest-VolumeQuotaClassesTest-2102805714-project-admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:34.801685 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 957/1893] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:31:34 2022] POST /identity/v3/projects => generated 354 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:31:34.808192 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1702d10-ad30-4283-83f5-c482ba0b3bc3 tempest-VolumeQuotaClassesTest-2102805714 tempest-VolumeQuotaClassesTest-2102805714-project-admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:34.825972 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1702d10-ad30-4283-83f5-c482ba0b3bc3 tempest-VolumeQuotaClassesTest-2102805714 tempest-VolumeQuotaClassesTest-2102805714-project-admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:34.907940 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec52b33f-c446-48ce-acfa-22772d9b847a tempest-VolumeQuotaClassesTest-2102805714 tempest-VolumeQuotaClassesTest-2102805714-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e271ace6a07409086817e93573c49af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad6b94324205456687f28eb3ff9c24c7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:34.908658 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52b33f-c446-48ce-acfa-22772d9b847a tempest-VolumeQuotaClassesTest-2102805714 tempest-VolumeQuotaClassesTest-2102805714-project-admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:34.909100 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52b33f-c446-48ce-acfa-22772d9b847a tempest-VolumeQuotaClassesTest-2102805714 tempest-VolumeQuotaClassesTest-2102805714-project-admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:34.909478 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52b33f-c446-48ce-acfa-22772d9b847a tempest-VolumeQuotaClassesTest-2102805714 tempest-VolumeQuotaClassesTest-2102805714-project-admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:34.911098 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec52b33f-c446-48ce-acfa-22772d9b847a tempest-VolumeQuotaClassesTest-2102805714 tempest-VolumeQuotaClassesTest-2102805714-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:31:34.911098 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec52b33f-c446-48ce-acfa-22772d9b847a tempest-VolumeQuotaClassesTest-2102805714 tempest-VolumeQuotaClassesTest-2102805714-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:34.916446 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec52b33f-c446-48ce-acfa-22772d9b847a tempest-VolumeQuotaClassesTest-2102805714 tempest-VolumeQuotaClassesTest-2102805714-project-admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:34.940182 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 937/1894] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:31:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:31:35.020205 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65fba7-e827-4144-ad8b-085bb702ec76 tempest-VolumeQuotaClassesTest-2102805714 tempest-VolumeQuotaClassesTest-2102805714-project-admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:35.032021 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65fba7-e827-4144-ad8b-085bb702ec76 tempest-VolumeQuotaClassesTest-2102805714 tempest-VolumeQuotaClassesTest-2102805714-project-admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:35.037207 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346c6e2-289e-4333-8e18-a1e0fa69e73a tempest-VolumeQuotaClassesTest-2102805714 tempest-VolumeQuotaClassesTest-2102805714-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e271ace6a07409086817e93573c49af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad6b94324205456687f28eb3ff9c24c7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:35.038596 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346c6e2-289e-4333-8e18-a1e0fa69e73a tempest-VolumeQuotaClassesTest-2102805714 tempest-VolumeQuotaClassesTest-2102805714-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:35.039014 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346c6e2-289e-4333-8e18-a1e0fa69e73a tempest-VolumeQuotaClassesTest-2102805714 tempest-VolumeQuotaClassesTest-2102805714-project-admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:35.039426 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346c6e2-289e-4333-8e18-a1e0fa69e73a tempest-VolumeQuotaClassesTest-2102805714 tempest-VolumeQuotaClassesTest-2102805714-project-admin] PATH_INFO: `/v3/projects/f1e6891e3f7b4414a309822e22d5dc74` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:35.041468 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346c6e2-289e-4333-8e18-a1e0fa69e73a tempest-VolumeQuotaClassesTest-2102805714 tempest-VolumeQuotaClassesTest-2102805714-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f1e6891e3f7b4414a309822e22d5dc74)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:35.043994 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346c6e2-289e-4333-8e18-a1e0fa69e73a tempest-VolumeQuotaClassesTest-2102805714 tempest-VolumeQuotaClassesTest-2102805714-project-admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:35.187371 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-8346c6e2-289e-4333-8e18-a1e0fa69e73a tempest-VolumeQuotaClassesTest-2102805714 tempest-VolumeQuotaClassesTest-2102805714-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1e6891e3f7b4414a309822e22d5dc74', 'request_id': 'req-8346c6e2-289e-4333-8e18-a1e0fa69e73a'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:35.194031 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-8346c6e2-289e-4333-8e18-a1e0fa69e73a tempest-VolumeQuotaClassesTest-2102805714 tempest-VolumeQuotaClassesTest-2102805714-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1e6891e3f7b4414a309822e22d5dc74', 'request_id': 'req-8346c6e2-289e-4333-8e18-a1e0fa69e73a'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:35.201279 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-8346c6e2-289e-4333-8e18-a1e0fa69e73a tempest-VolumeQuotaClassesTest-2102805714 tempest-VolumeQuotaClassesTest-2102805714-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1e6891e3f7b4414a309822e22d5dc74', 'request_id': 'req-8346c6e2-289e-4333-8e18-a1e0fa69e73a'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:35.289391 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-8346c6e2-289e-4333-8e18-a1e0fa69e73a tempest-VolumeQuotaClassesTest-2102805714 tempest-VolumeQuotaClassesTest-2102805714-project-admin] The token cache is being invalidate because project f1e6891e3f7b4414a309822e22d5dc74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:31:35.289792 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-8346c6e2-289e-4333-8e18-a1e0fa69e73a tempest-VolumeQuotaClassesTest-2102805714 tempest-VolumeQuotaClassesTest-2102805714-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:35.290647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-8346c6e2-289e-4333-8e18-a1e0fa69e73a tempest-VolumeQuotaClassesTest-2102805714 tempest-VolumeQuotaClassesTest-2102805714-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:35.292357 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 958/1895] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:31:35 2022] DELETE /identity/v3/projects/f1e6891e3f7b4414a309822e22d5dc74 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:35.890761 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:35.968819 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec52b33f-c446-48ce-acfa-22772d9b847a tempest-VolumeQuotaClassesTest-2102805714 tempest-VolumeQuotaClassesTest-2102805714-project-admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:35.971193 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-ec52b33f-c446-48ce-acfa-22772d9b847a tempest-VolumeQuotaClassesTest-2102805714 tempest-VolumeQuotaClassesTest-2102805714-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:35.998980 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:36.059701 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec52b33f-c446-48ce-acfa-22772d9b847a tempest-VolumeQuotaClassesTest-2102805714 tempest-VolumeQuotaClassesTest-2102805714-project-admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:36.123134 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a53164-f50b-4e35-a95c-ee7b4283f0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:36.123791 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a53164-f50b-4e35-a95c-ee7b4283f0c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:36.124176 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a53164-f50b-4e35-a95c-ee7b4283f0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:36.124563 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a53164-f50b-4e35-a95c-ee7b4283f0c0 admin admin] PATH_INFO: `/v3/users/43b5852d6657448fbd510f504448faea` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:36.125895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a53164-f50b-4e35-a95c-ee7b4283f0c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43b5852d6657448fbd510f504448faea)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:36.128045 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a53164-f50b-4e35-a95c-ee7b4283f0c0 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:36.364785 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-13a53164-f50b-4e35-a95c-ee7b4283f0c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43b5852d6657448fbd510f504448faea'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:36.380358 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-13a53164-f50b-4e35-a95c-ee7b4283f0c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43b5852d6657448fbd510f504448faea'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:36.381141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-13a53164-f50b-4e35-a95c-ee7b4283f0c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43b5852d6657448fbd510f504448faea'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:36.394567 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-13a53164-f50b-4e35-a95c-ee7b4283f0c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43b5852d6657448fbd510f504448faea'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:36.395271 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-13a53164-f50b-4e35-a95c-ee7b4283f0c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43b5852d6657448fbd510f504448faea'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:36.443529 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 938/1896] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:31:35 2022] DELETE /identity/v3/users/43b5852d6657448fbd510f504448faea => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:36.448035 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13ffad-abe8-4e6e-aa5c-105950e82676 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:36.448529 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13ffad-abe8-4e6e-aa5c-105950e82676 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:36.448659 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13ffad-abe8-4e6e-aa5c-105950e82676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:36.469406 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-fb13ffad-abe8-4e6e-aa5c-105950e82676 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:36.471268 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-fb13ffad-abe8-4e6e-aa5c-105950e82676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:36.540578 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 959/1897] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:36.550451 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a53164-f50b-4e35-a95c-ee7b4283f0c0 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:36.551892 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-13a53164-f50b-4e35-a95c-ee7b4283f0c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:36.616009 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a53164-f50b-4e35-a95c-ee7b4283f0c0 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:36.619933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d32e5-b107-4bdf-946a-6e6cea077167 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:36.620580 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d32e5-b107-4bdf-946a-6e6cea077167 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:36.620862 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d32e5-b107-4bdf-946a-6e6cea077167 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:36.621138 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d32e5-b107-4bdf-946a-6e6cea077167 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:36.634233 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60d32e5-b107-4bdf-946a-6e6cea077167 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:36.634233 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60d32e5-b107-4bdf-946a-6e6cea077167 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:36.643577 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 939/1898] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:31:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 15:31:36.747626 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:37.116534 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13ffad-abe8-4e6e-aa5c-105950e82676 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:37.118043 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-fb13ffad-abe8-4e6e-aa5c-105950e82676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:37.132173 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13ffad-abe8-4e6e-aa5c-105950e82676 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:37.137512 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7662da9-88e3-424f-bb8a-4f80b0ca6534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:37.138482 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7662da9-88e3-424f-bb8a-4f80b0ca6534 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:37.138967 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7662da9-88e3-424f-bb8a-4f80b0ca6534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:37.139459 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7662da9-88e3-424f-bb8a-4f80b0ca6534 admin admin] PATH_INFO: `/v3/projects/e531255ed991445f90e3285f48158313` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:37.141362 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7662da9-88e3-424f-bb8a-4f80b0ca6534 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e531255ed991445f90e3285f48158313)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:37.143602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7662da9-88e3-424f-bb8a-4f80b0ca6534 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:37.302580 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c7662da9-88e3-424f-bb8a-4f80b0ca6534 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e531255ed991445f90e3285f48158313', 'request_id': 'req-c7662da9-88e3-424f-bb8a-4f80b0ca6534'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:37.308682 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c7662da9-88e3-424f-bb8a-4f80b0ca6534 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e531255ed991445f90e3285f48158313', 'request_id': 'req-c7662da9-88e3-424f-bb8a-4f80b0ca6534'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:37.314678 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c7662da9-88e3-424f-bb8a-4f80b0ca6534 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e531255ed991445f90e3285f48158313', 'request_id': 'req-c7662da9-88e3-424f-bb8a-4f80b0ca6534'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:37.427577 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c7662da9-88e3-424f-bb8a-4f80b0ca6534 admin admin] The token cache is being invalidate because project e531255ed991445f90e3285f48158313 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:31:37.427804 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c7662da9-88e3-424f-bb8a-4f80b0ca6534 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:37.428518 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c7662da9-88e3-424f-bb8a-4f80b0ca6534 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:37.430040 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 960/1899] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:31:37 2022] DELETE /identity/v3/projects/e531255ed991445f90e3285f48158313 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:37.433895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d32e5-b107-4bdf-946a-6e6cea077167 service neutron] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:37.443626 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d32e5-b107-4bdf-946a-6e6cea077167 service neutron] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:37.511962 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023ac8d-d307-45d2-bb63-b18daf735b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:37.512639 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023ac8d-d307-45d2-bb63-b18daf735b28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:37.512925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023ac8d-d307-45d2-bb63-b18daf735b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:37.513207 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023ac8d-d307-45d2-bb63-b18daf735b28 admin admin] PATH_INFO: `/v3/users/2e271ace6a07409086817e93573c49af` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:37.514395 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d023ac8d-d307-45d2-bb63-b18daf735b28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e271ace6a07409086817e93573c49af)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:37.516374 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d023ac8d-d307-45d2-bb63-b18daf735b28 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:37.687874 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-d023ac8d-d307-45d2-bb63-b18daf735b28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e271ace6a07409086817e93573c49af'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:37.696353 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-d023ac8d-d307-45d2-bb63-b18daf735b28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e271ace6a07409086817e93573c49af'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:37.697170 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-d023ac8d-d307-45d2-bb63-b18daf735b28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e271ace6a07409086817e93573c49af'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:37.712753 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-d023ac8d-d307-45d2-bb63-b18daf735b28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e271ace6a07409086817e93573c49af'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:37.713482 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-d023ac8d-d307-45d2-bb63-b18daf735b28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e271ace6a07409086817e93573c49af'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:37.813188 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 940/1900] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:31:37 2022] DELETE /identity/v3/users/2e271ace6a07409086817e93573c49af => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:38.267015 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:38.344627 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7662da9-88e3-424f-bb8a-4f80b0ca6534 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:38.347266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-c7662da9-88e3-424f-bb8a-4f80b0ca6534 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:38.362564 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7662da9-88e3-424f-bb8a-4f80b0ca6534 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:38.447729 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d061115-79e5-4f17-907e-ab72c5d3a410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:38.448451 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d061115-79e5-4f17-907e-ab72c5d3a410 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:38.448830 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d061115-79e5-4f17-907e-ab72c5d3a410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:38.449007 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d061115-79e5-4f17-907e-ab72c5d3a410 admin admin] PATH_INFO: `/v3/projects/ad6b94324205456687f28eb3ff9c24c7` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:38.450454 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d061115-79e5-4f17-907e-ab72c5d3a410 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad6b94324205456687f28eb3ff9c24c7)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:38.452155 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:38.456664 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d061115-79e5-4f17-907e-ab72c5d3a410 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:38.570394 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-4d061115-79e5-4f17-907e-ab72c5d3a410 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad6b94324205456687f28eb3ff9c24c7', 'request_id': 'req-4d061115-79e5-4f17-907e-ab72c5d3a410'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:38.576794 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-4d061115-79e5-4f17-907e-ab72c5d3a410 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad6b94324205456687f28eb3ff9c24c7', 'request_id': 'req-4d061115-79e5-4f17-907e-ab72c5d3a410'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:38.582741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-4d061115-79e5-4f17-907e-ab72c5d3a410 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad6b94324205456687f28eb3ff9c24c7', 'request_id': 'req-4d061115-79e5-4f17-907e-ab72c5d3a410'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:38.689748 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-4d061115-79e5-4f17-907e-ab72c5d3a410 admin admin] The token cache is being invalidate because project ad6b94324205456687f28eb3ff9c24c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:31:38.690048 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-4d061115-79e5-4f17-907e-ab72c5d3a410 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:38.690921 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-4d061115-79e5-4f17-907e-ab72c5d3a410 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:38.692648 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 961/1901] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:31:38 2022] DELETE /identity/v3/projects/ad6b94324205456687f28eb3ff9c24c7 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:38.703048 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef145cd8-cd3b-491c-ba81-005aea692f4c None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:38.703048 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef145cd8-cd3b-491c-ba81-005aea692f4c None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:38.703048 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef145cd8-cd3b-491c-ba81-005aea692f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:38.718872 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-ef145cd8-cd3b-491c-ba81-005aea692f4c None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:38.720114 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-ef145cd8-cd3b-491c-ba81-005aea692f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:38.778417 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 941/1902] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:31:38.786305 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63dad2b-8014-41ea-857c-bef01d06c96a None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:38.786595 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63dad2b-8014-41ea-857c-bef01d06c96a None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:38.786878 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63dad2b-8014-41ea-857c-bef01d06c96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:38.807092 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-c63dad2b-8014-41ea-857c-bef01d06c96a None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:38.808352 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-c63dad2b-8014-41ea-857c-bef01d06c96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:38.817527 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 962/1903] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:38.821730 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef145cd8-cd3b-491c-ba81-005aea692f4c None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:38.831586 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef145cd8-cd3b-491c-ba81-005aea692f4c None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:38.836515 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979c45b1-2442-40a2-9604-c8c561ac59ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:38.837179 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c45b1-2442-40a2-9604-c8c561ac59ef admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:38.837437 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c45b1-2442-40a2-9604-c8c561ac59ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:38.837723 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c45b1-2442-40a2-9604-c8c561ac59ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:38.838427 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979c45b1-2442-40a2-9604-c8c561ac59ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:31:38.838740 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979c45b1-2442-40a2-9604-c8c561ac59ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:38.840380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979c45b1-2442-40a2-9604-c8c561ac59ef admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:38.856923 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 942/1904] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:31:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:31:38.858100 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63dad2b-8014-41ea-857c-bef01d06c96a None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:38.868379 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63dad2b-8014-41ea-857c-bef01d06c96a None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:38.873326 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa02178-1858-464b-a858-2854f6c78f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:38.874011 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa02178-1858-464b-a858-2854f6c78f4d admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:38.874360 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa02178-1858-464b-a858-2854f6c78f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:38.874705 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa02178-1858-464b-a858-2854f6c78f4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:38.875432 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa02178-1858-464b-a858-2854f6c78f4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:38.877514 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa02178-1858-464b-a858-2854f6c78f4d admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:38.973983 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 963/1905] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:31:38 2022] POST /identity/v3/projects => generated 391 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:31:38.978177 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979c45b1-2442-40a2-9604-c8c561ac59ef admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:38.989252 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979c45b1-2442-40a2-9604-c8c561ac59ef admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:39.055609 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f687963-850d-4974-880a-62d35d485634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:39.055609 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f687963-850d-4974-880a-62d35d485634 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:39.055609 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f687963-850d-4974-880a-62d35d485634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:39.055609 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f687963-850d-4974-880a-62d35d485634 admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:39.055927 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f687963-850d-4974-880a-62d35d485634 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:39.057199 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f687963-850d-4974-880a-62d35d485634 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:39.136228 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 943/1906] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:31:38 2022] POST /identity/v3/users => generated 296 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:31:39.136228 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f687963-850d-4974-880a-62d35d485634 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:39.137534 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa02178-1858-464b-a858-2854f6c78f4d admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:39.145904 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-7aa02178-1858-464b-a858-2854f6c78f4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:39.147042 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f687963-850d-4974-880a-62d35d485634 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:39.152916 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8550cc6e-d651-4866-b5b2-c9f31cddae52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:39.153667 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550cc6e-d651-4866-b5b2-c9f31cddae52 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:39.154091 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550cc6e-d651-4866-b5b2-c9f31cddae52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:39.154431 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550cc6e-d651-4866-b5b2-c9f31cddae52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:39.160306 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8550cc6e-d651-4866-b5b2-c9f31cddae52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:39.160306 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8550cc6e-d651-4866-b5b2-c9f31cddae52 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:39.163870 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa02178-1858-464b-a858-2854f6c78f4d admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:39.170991 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdec0c71-c537-417f-8262-0bd88b6f2be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:39.172012 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec0c71-c537-417f-8262-0bd88b6f2be8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:39.174352 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec0c71-c537-417f-8262-0bd88b6f2be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:39.174799 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 944/1907] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:31:39.174985 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec0c71-c537-417f-8262-0bd88b6f2be8 admin admin] PATH_INFO: `/v3/users/2c81dc9230574ce88c7828917e7bb660` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:39.175811 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8550cc6e-d651-4866-b5b2-c9f31cddae52 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:39.177550 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdec0c71-c537-417f-8262-0bd88b6f2be8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c81dc9230574ce88c7828917e7bb660)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:39.182343 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdec0c71-c537-417f-8262-0bd88b6f2be8 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:39.189385 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8550cc6e-d651-4866-b5b2-c9f31cddae52 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:39.203730 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bcb701-51b3-4831-b8ad-f9540f18b2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:39.205427 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bcb701-51b3-4831-b8ad-f9540f18b2ef admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:39.212295 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bcb701-51b3-4831-b8ad-f9540f18b2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:39.212295 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bcb701-51b3-4831-b8ad-f9540f18b2ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:39.212295 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bcb701-51b3-4831-b8ad-f9540f18b2ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:39.212295 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bcb701-51b3-4831-b8ad-f9540f18b2ef admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:39.224102 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 945/1908] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:31:39.228418 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bcb701-51b3-4831-b8ad-f9540f18b2ef admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:39.242519 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bcb701-51b3-4831-b8ad-f9540f18b2ef admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:39.248609 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0df15f-6a8d-439c-ae25-b47cdaa818c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:39.249287 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0df15f-6a8d-439c-ae25-b47cdaa818c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:39.249682 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0df15f-6a8d-439c-ae25-b47cdaa818c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:39.249835 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0df15f-6a8d-439c-ae25-b47cdaa818c5 admin admin] PATH_INFO: `/v3/projects/2f4668b357ae4baf957680eeacf627fe/users/11e69daf4f0b40bfb2a18a259b70c8cb/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:39.267651 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0df15f-6a8d-439c-ae25-b47cdaa818c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f4668b357ae4baf957680eeacf627fe, user_id=11e69daf4f0b40bfb2a18a259b70c8cb, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:39.271326 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0df15f-6a8d-439c-ae25-b47cdaa818c5 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:39.347742 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 946/1909] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:31:39 2022] PUT /identity/v3/projects/2f4668b357ae4baf957680eeacf627fe/users/11e69daf4f0b40bfb2a18a259b70c8cb/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:39.357984 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448b03d-5fb7-47dc-bee1-34a2faf285b5 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:39.358281 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448b03d-5fb7-47dc-bee1-34a2faf285b5 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:39.358566 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448b03d-5fb7-47dc-bee1-34a2faf285b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:39.386162 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-f448b03d-5fb7-47dc-bee1-34a2faf285b5 None None] MFA Rules not processed for user `11e69daf4f0b40bfb2a18a259b70c8cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:39.387389 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-f448b03d-5fb7-47dc-bee1-34a2faf285b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:39.399899 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-bdec0c71-c537-417f-8262-0bd88b6f2be8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c81dc9230574ce88c7828917e7bb660'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:39.400579 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-bdec0c71-c537-417f-8262-0bd88b6f2be8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c81dc9230574ce88c7828917e7bb660'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:39.413554 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-bdec0c71-c537-417f-8262-0bd88b6f2be8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c81dc9230574ce88c7828917e7bb660'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:39.464815 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 947/1910] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:39 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:31:39.469163 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f448b03d-5fb7-47dc-bee1-34a2faf285b5 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:39.471226 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-f448b03d-5fb7-47dc-bee1-34a2faf285b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:39.511863 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-bdec0c71-c537-417f-8262-0bd88b6f2be8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c81dc9230574ce88c7828917e7bb660'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:39.521428 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-bdec0c71-c537-417f-8262-0bd88b6f2be8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c81dc9230574ce88c7828917e7bb660'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:39.532086 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 964/1911] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:31:39 2022] DELETE /identity/v3/users/2c81dc9230574ce88c7828917e7bb660 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:31:39.537573 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6829f-3f50-4b31-8286-a2fbb3d2be95 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:39.537973 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6829f-3f50-4b31-8286-a2fbb3d2be95 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:39.538419 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6829f-3f50-4b31-8286-a2fbb3d2be95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:39.544045 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f448b03d-5fb7-47dc-bee1-34a2faf285b5 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:39.560126 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-07e6829f-3f50-4b31-8286-a2fbb3d2be95 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:39.561539 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-07e6829f-3f50-4b31-8286-a2fbb3d2be95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:39.613102 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1efaf51-1dbc-467b-a019-39b759e027c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:39.616555 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1efaf51-1dbc-467b-a019-39b759e027c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:39.616555 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1efaf51-1dbc-467b-a019-39b759e027c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:39.616555 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1efaf51-1dbc-467b-a019-39b759e027c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:39.616555 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1efaf51-1dbc-467b-a019-39b759e027c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:39.617432 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1efaf51-1dbc-467b-a019-39b759e027c7 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:39.650511 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 965/1912] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:39.664396 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e6829f-3f50-4b31-8286-a2fbb3d2be95 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:39.666088 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-07e6829f-3f50-4b31-8286-a2fbb3d2be95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:39.720267 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 948/1913] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:31:39 2022] POST /identity/v3/projects => generated 393 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:31:39.725292 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1efaf51-1dbc-467b-a019-39b759e027c7 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:39.742829 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e6829f-3f50-4b31-8286-a2fbb3d2be95 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:39.832634 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fdeebd-1009-4954-9df5-7936bd82cf84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:39.833279 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fdeebd-1009-4954-9df5-7936bd82cf84 service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:39.833567 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fdeebd-1009-4954-9df5-7936bd82cf84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:39.833844 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fdeebd-1009-4954-9df5-7936bd82cf84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:39.862307 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1efaf51-1dbc-467b-a019-39b759e027c7 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:39.868153 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddcab8a-ac5a-40a3-968c-d33fed84d47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:39.868850 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddcab8a-ac5a-40a3-968c-d33fed84d47b admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:39.869065 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddcab8a-ac5a-40a3-968c-d33fed84d47b admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:39.869349 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddcab8a-ac5a-40a3-968c-d33fed84d47b admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:39.870028 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddcab8a-ac5a-40a3-968c-d33fed84d47b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:39.872003 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddcab8a-ac5a-40a3-968c-d33fed84d47b admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:39.908908 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fdeebd-1009-4954-9df5-7936bd82cf84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:39.910841 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fdeebd-1009-4954-9df5-7936bd82cf84 service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:39.920969 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 966/1914] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:31:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:31:39.968911 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 949/1915] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:31:39 2022] POST /identity/v3/users => generated 303 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:31:39.970234 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fdeebd-1009-4954-9df5-7936bd82cf84 service neutron] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:39.983126 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fdeebd-1009-4954-9df5-7936bd82cf84 service neutron] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:39.988808 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b647c-9a4c-470d-8daf-b6fee9434163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:39.989520 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b647c-9a4c-470d-8daf-b6fee9434163 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:39.989942 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b647c-9a4c-470d-8daf-b6fee9434163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:39.990294 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b647c-9a4c-470d-8daf-b6fee9434163 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:39.991537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33b647c-9a4c-470d-8daf-b6fee9434163 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:39.993809 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33b647c-9a4c-470d-8daf-b6fee9434163 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:40.011732 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 967/1916] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:31:40.013005 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddcab8a-ac5a-40a3-968c-d33fed84d47b admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:40.028635 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddcab8a-ac5a-40a3-968c-d33fed84d47b admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:40.035326 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe150af9-6942-4f3d-bcb9-d157fa00e748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:40.036574 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe150af9-6942-4f3d-bcb9-d157fa00e748 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:40.036884 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe150af9-6942-4f3d-bcb9-d157fa00e748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:40.037189 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe150af9-6942-4f3d-bcb9-d157fa00e748 admin admin] PATH_INFO: `/v3/projects/ed2115d8151647cbafc5d265a84dfbc0/users/4e1a532237bb42cb9f287efa0af360a6/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:40.054110 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe150af9-6942-4f3d-bcb9-d157fa00e748 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed2115d8151647cbafc5d265a84dfbc0, user_id=4e1a532237bb42cb9f287efa0af360a6, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:40.056862 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe150af9-6942-4f3d-bcb9-d157fa00e748 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:40.126931 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 950/1917] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:31:40 2022] PUT /identity/v3/projects/ed2115d8151647cbafc5d265a84dfbc0/users/4e1a532237bb42cb9f287efa0af360a6/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:40.131478 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b647c-9a4c-470d-8daf-b6fee9434163 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:40.146400 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b647c-9a4c-470d-8daf-b6fee9434163 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:40.272312 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:40.281045 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176291e7-8a22-427c-bfb8-f52dbba7b268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:40.282588 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176291e7-8a22-427c-bfb8-f52dbba7b268 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:40.283050 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176291e7-8a22-427c-bfb8-f52dbba7b268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:40.283358 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176291e7-8a22-427c-bfb8-f52dbba7b268 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:40.284083 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176291e7-8a22-427c-bfb8-f52dbba7b268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:40.285788 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176291e7-8a22-427c-bfb8-f52dbba7b268 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:40.299170 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 968/1918] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:31:40 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 15:31:40.303603 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe150af9-6942-4f3d-bcb9-d157fa00e748 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:40.323848 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe150af9-6942-4f3d-bcb9-d157fa00e748 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:40.337030 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcb1d0c-9617-45a1-95dd-8c673eeebb91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:40.337821 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb1d0c-9617-45a1-95dd-8c673eeebb91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:40.338103 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb1d0c-9617-45a1-95dd-8c673eeebb91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:40.338410 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb1d0c-9617-45a1-95dd-8c673eeebb91 admin admin] PATH_INFO: `/v3/projects/ed2115d8151647cbafc5d265a84dfbc0/users/4e1a532237bb42cb9f287efa0af360a6/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:40.340754 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcb1d0c-9617-45a1-95dd-8c673eeebb91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed2115d8151647cbafc5d265a84dfbc0, user_id=4e1a532237bb42cb9f287efa0af360a6, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:40.343374 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcb1d0c-9617-45a1-95dd-8c673eeebb91 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:40.356674 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 951/1919] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:31:40 2022] PUT /identity/v3/projects/ed2115d8151647cbafc5d265a84dfbc0/users/4e1a532237bb42cb9f287efa0af360a6/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:40.369361 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac282499-e2a5-49a0-beeb-2b4362fb9dfa None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:40.369361 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac282499-e2a5-49a0-beeb-2b4362fb9dfa None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:40.369361 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac282499-e2a5-49a0-beeb-2b4362fb9dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:40.406934 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-ac282499-e2a5-49a0-beeb-2b4362fb9dfa None None] MFA Rules not processed for user `4e1a532237bb42cb9f287efa0af360a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:40.408915 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-ac282499-e2a5-49a0-beeb-2b4362fb9dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:40.417931 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:40.492089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcb1d0c-9617-45a1-95dd-8c673eeebb91 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:40.494116 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-8fcb1d0c-9617-45a1-95dd-8c673eeebb91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:40.549734 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 969/1920] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:40 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:40.558989 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac282499-e2a5-49a0-beeb-2b4362fb9dfa None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:40.562206 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-ac282499-e2a5-49a0-beeb-2b4362fb9dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:40.635364 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcb1d0c-9617-45a1-95dd-8c673eeebb91 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:40.641104 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1519e-76da-43b5-98dc-117fde6c80f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:40.641395 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1519e-76da-43b5-98dc-117fde6c80f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:40.646362 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1519e-76da-43b5-98dc-117fde6c80f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:40.646362 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1519e-76da-43b5-98dc-117fde6c80f6 admin admin] PATH_INFO: `/v3/projects/eef07a3eee044b9aa54fc568cb66cc0f` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:40.646362 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1519e-76da-43b5-98dc-117fde6c80f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eef07a3eee044b9aa54fc568cb66cc0f)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:40.646362 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1519e-76da-43b5-98dc-117fde6c80f6 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:40.669776 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac282499-e2a5-49a0-beeb-2b4362fb9dfa None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:40.687582 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f866fec-3d52-4cca-9598-7730b5d9d98a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:40.688324 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f866fec-3d52-4cca-9598-7730b5d9d98a service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:40.688701 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f866fec-3d52-4cca-9598-7730b5d9d98a service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:40.689255 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f866fec-3d52-4cca-9598-7730b5d9d98a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:40.712532 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f866fec-3d52-4cca-9598-7730b5d9d98a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:40.715879 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f866fec-3d52-4cca-9598-7730b5d9d98a service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:40.736723 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 970/1921] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:31:40 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 15:31:40.790880 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f866fec-3d52-4cca-9598-7730b5d9d98a service cinder] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:40.792562 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-3f866fec-3d52-4cca-9598-7730b5d9d98a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:40.812457 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f866fec-3d52-4cca-9598-7730b5d9d98a service cinder] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:40.821205 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-fbb1519e-76da-43b5-98dc-117fde6c80f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eef07a3eee044b9aa54fc568cb66cc0f', 'request_id': 'req-fbb1519e-76da-43b5-98dc-117fde6c80f6'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:40.830079 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-fbb1519e-76da-43b5-98dc-117fde6c80f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eef07a3eee044b9aa54fc568cb66cc0f', 'request_id': 'req-fbb1519e-76da-43b5-98dc-117fde6c80f6'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:40.911457 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2701d5-46de-4413-851d-2330f462f315 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:40.912335 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2701d5-46de-4413-851d-2330f462f315 service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:40.912715 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2701d5-46de-4413-851d-2330f462f315 service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:40.913965 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2701d5-46de-4413-851d-2330f462f315 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:40.925382 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-fbb1519e-76da-43b5-98dc-117fde6c80f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eef07a3eee044b9aa54fc568cb66cc0f', 'request_id': 'req-fbb1519e-76da-43b5-98dc-117fde6c80f6'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:40.948104 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-fbb1519e-76da-43b5-98dc-117fde6c80f6 admin admin] The token cache is being invalidate because project eef07a3eee044b9aa54fc568cb66cc0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:31:40.948568 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-fbb1519e-76da-43b5-98dc-117fde6c80f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:40.950303 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-fbb1519e-76da-43b5-98dc-117fde6c80f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:40.956251 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 952/1922] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:31:40 2022] DELETE /identity/v3/projects/eef07a3eee044b9aa54fc568cb66cc0f => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:40.960636 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1519e-76da-43b5-98dc-117fde6c80f6 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:40.963686 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-fbb1519e-76da-43b5-98dc-117fde6c80f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:41.036780 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2701d5-46de-4413-851d-2330f462f315 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:41.038553 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2701d5-46de-4413-851d-2330f462f315 service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:41.080504 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 971/1923] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:31:40 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:31:41.092566 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1519e-76da-43b5-98dc-117fde6c80f6 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:41.104774 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a6e14-d785-45ef-b667-25ef8e659cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:41.105466 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a6e14-d785-45ef-b667-25ef8e659cfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:41.105756 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a6e14-d785-45ef-b667-25ef8e659cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:41.106040 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a6e14-d785-45ef-b667-25ef8e659cfb admin admin] PATH_INFO: `/v3/users/0b25f0dc047143da963336ed1fa16f6d` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:41.107245 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763a6e14-d785-45ef-b667-25ef8e659cfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b25f0dc047143da963336ed1fa16f6d)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:41.109284 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763a6e14-d785-45ef-b667-25ef8e659cfb admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:41.315839 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-763a6e14-d785-45ef-b667-25ef8e659cfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b25f0dc047143da963336ed1fa16f6d'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:41.323314 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-763a6e14-d785-45ef-b667-25ef8e659cfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b25f0dc047143da963336ed1fa16f6d'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:41.324104 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-763a6e14-d785-45ef-b667-25ef8e659cfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b25f0dc047143da963336ed1fa16f6d'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:41.336477 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-763a6e14-d785-45ef-b667-25ef8e659cfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b25f0dc047143da963336ed1fa16f6d'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:41.337134 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-763a6e14-d785-45ef-b667-25ef8e659cfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b25f0dc047143da963336ed1fa16f6d'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:41.409658 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 953/1924] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:31:40 2022] DELETE /identity/v3/users/0b25f0dc047143da963336ed1fa16f6d => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:41.439415 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:41.913801 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2701d5-46de-4413-851d-2330f462f315 service cinder] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:41.928124 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2701d5-46de-4413-851d-2330f462f315 service cinder] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:42.005069 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a9f3fe-b9bd-4959-85a0-72c7ed9add36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:42.005069 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9f3fe-b9bd-4959-85a0-72c7ed9add36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:42.005069 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9f3fe-b9bd-4959-85a0-72c7ed9add36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:42.005069 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9f3fe-b9bd-4959-85a0-72c7ed9add36 admin admin] PATH_INFO: `/v3/projects/fcb729706f6c4f908886cd7271ce808d` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:42.005462 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a9f3fe-b9bd-4959-85a0-72c7ed9add36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcb729706f6c4f908886cd7271ce808d)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:42.008581 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a9f3fe-b9bd-4959-85a0-72c7ed9add36 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:42.130795 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:42.157836 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-70a9f3fe-b9bd-4959-85a0-72c7ed9add36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcb729706f6c4f908886cd7271ce808d', 'request_id': 'req-70a9f3fe-b9bd-4959-85a0-72c7ed9add36'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:42.169238 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-70a9f3fe-b9bd-4959-85a0-72c7ed9add36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcb729706f6c4f908886cd7271ce808d', 'request_id': 'req-70a9f3fe-b9bd-4959-85a0-72c7ed9add36'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:42.177534 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-70a9f3fe-b9bd-4959-85a0-72c7ed9add36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcb729706f6c4f908886cd7271ce808d', 'request_id': 'req-70a9f3fe-b9bd-4959-85a0-72c7ed9add36'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:42.278248 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-70a9f3fe-b9bd-4959-85a0-72c7ed9add36 admin admin] The token cache is being invalidate because project fcb729706f6c4f908886cd7271ce808d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:31:42.282889 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-70a9f3fe-b9bd-4959-85a0-72c7ed9add36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:42.282889 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-70a9f3fe-b9bd-4959-85a0-72c7ed9add36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:42.282889 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 972/1925] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:31:41 2022] DELETE /identity/v3/projects/fcb729706f6c4f908886cd7271ce808d => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:31:42.286775 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31bc6f-6ad6-4708-9886-e9ca4893c4eb None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:42.287175 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31bc6f-6ad6-4708-9886-e9ca4893c4eb None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:42.287533 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31bc6f-6ad6-4708-9886-e9ca4893c4eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:42.302770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:42.311389 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-ee31bc6f-6ad6-4708-9886-e9ca4893c4eb None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:42.312739 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-ee31bc6f-6ad6-4708-9886-e9ca4893c4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:42.408216 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 954/1926] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:42.414461 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886e6c4-8d7d-4916-9281-ee7100a305d9 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:42.414756 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886e6c4-8d7d-4916-9281-ee7100a305d9 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:42.415043 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886e6c4-8d7d-4916-9281-ee7100a305d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:42.440625 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-9886e6c4-8d7d-4916-9281-ee7100a305d9 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:42.441856 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-9886e6c4-8d7d-4916-9281-ee7100a305d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:42.451405 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 973/1927] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:42.456570 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee31bc6f-6ad6-4708-9886-e9ca4893c4eb None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:42.466842 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee31bc6f-6ad6-4708-9886-e9ca4893c4eb None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:42.484995 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc2e295-47fb-4aba-ae39-a5ebe7d111dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:42.485873 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2e295-47fb-4aba-ae39-a5ebe7d111dd admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:42.486153 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2e295-47fb-4aba-ae39-a5ebe7d111dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:42.486602 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2e295-47fb-4aba-ae39-a5ebe7d111dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:42.491847 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc2e295-47fb-4aba-ae39-a5ebe7d111dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:31:42.492294 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc2e295-47fb-4aba-ae39-a5ebe7d111dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:42.494060 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc2e295-47fb-4aba-ae39-a5ebe7d111dd admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:42.507051 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 955/1928] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:31:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:31:42.514197 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9886e6c4-8d7d-4916-9281-ee7100a305d9 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:42.524518 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9886e6c4-8d7d-4916-9281-ee7100a305d9 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:42.530939 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd9d04e-7b6c-45ce-8538-ff6eade38b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:42.531625 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9d04e-7b6c-45ce-8538-ff6eade38b06 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:42.531919 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9d04e-7b6c-45ce-8538-ff6eade38b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:42.532833 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9d04e-7b6c-45ce-8538-ff6eade38b06 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:42.533535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd9d04e-7b6c-45ce-8538-ff6eade38b06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:42.535492 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd9d04e-7b6c-45ce-8538-ff6eade38b06 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:42.664594 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 974/1929] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:31:42 2022] POST /identity/v3/projects => generated 385 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:31:42.668922 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc2e295-47fb-4aba-ae39-a5ebe7d111dd admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:42.681620 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc2e295-47fb-4aba-ae39-a5ebe7d111dd admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:42.755427 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed37e0-472e-4ddf-a8b7-5f83af9ba9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:42.756389 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed37e0-472e-4ddf-a8b7-5f83af9ba9b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:42.757151 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed37e0-472e-4ddf-a8b7-5f83af9ba9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:42.757279 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed37e0-472e-4ddf-a8b7-5f83af9ba9b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:42.757889 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ed37e0-472e-4ddf-a8b7-5f83af9ba9b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:42.759995 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ed37e0-472e-4ddf-a8b7-5f83af9ba9b6 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:42.857077 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 956/1930] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:31:42 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:31:42.860448 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd9d04e-7b6c-45ce-8538-ff6eade38b06 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:42.871054 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd9d04e-7b6c-45ce-8538-ff6eade38b06 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:42.876251 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f9fd2-7cc9-4155-bf49-742bde39d8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:42.876903 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f9fd2-7cc9-4155-bf49-742bde39d8e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:42.877193 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f9fd2-7cc9-4155-bf49-742bde39d8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:42.877490 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f9fd2-7cc9-4155-bf49-742bde39d8e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:42.878199 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2f9fd2-7cc9-4155-bf49-742bde39d8e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:42.879840 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2f9fd2-7cc9-4155-bf49-742bde39d8e0 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:42.890161 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 975/1931] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:31:42.895409 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed37e0-472e-4ddf-a8b7-5f83af9ba9b6 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:42.905957 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed37e0-472e-4ddf-a8b7-5f83af9ba9b6 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:42.910553 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7173b33-47bd-4898-bf28-ac02daba7f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:42.911160 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7173b33-47bd-4898-bf28-ac02daba7f69 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:42.911445 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7173b33-47bd-4898-bf28-ac02daba7f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:42.911722 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7173b33-47bd-4898-bf28-ac02daba7f69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:42.912451 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7173b33-47bd-4898-bf28-ac02daba7f69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:42.914087 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7173b33-47bd-4898-bf28-ac02daba7f69 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:42.925326 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 957/1932] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:31:42.929851 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f9fd2-7cc9-4155-bf49-742bde39d8e0 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:42.940486 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f9fd2-7cc9-4155-bf49-742bde39d8e0 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:42.945069 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c531c3f-0a58-4582-9a5b-f16695f4e868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:42.945739 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c531c3f-0a58-4582-9a5b-f16695f4e868 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:42.946025 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c531c3f-0a58-4582-9a5b-f16695f4e868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:42.946341 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c531c3f-0a58-4582-9a5b-f16695f4e868 admin admin] PATH_INFO: `/v3/projects/43cb2fa3e8a1460f9a68979fb8e9299b/users/a8268ff3ab6248ea924648911dc6a1e6/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:42.960763 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c531c3f-0a58-4582-9a5b-f16695f4e868 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43cb2fa3e8a1460f9a68979fb8e9299b, user_id=a8268ff3ab6248ea924648911dc6a1e6, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:42.963447 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c531c3f-0a58-4582-9a5b-f16695f4e868 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:43.079278 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 976/1933] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:31:42 2022] PUT /identity/v3/projects/43cb2fa3e8a1460f9a68979fb8e9299b/users/a8268ff3ab6248ea924648911dc6a1e6/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:43.089939 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d86243-0ce5-4cd2-985d-517939091741 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:43.090226 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d86243-0ce5-4cd2-985d-517939091741 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:43.090534 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d86243-0ce5-4cd2-985d-517939091741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:43.120597 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-17d86243-0ce5-4cd2-985d-517939091741 None None] MFA Rules not processed for user `a8268ff3ab6248ea924648911dc6a1e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:43.121948 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-17d86243-0ce5-4cd2-985d-517939091741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:43.209249 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 958/1934] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:31:43.213986 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c531c3f-0a58-4582-9a5b-f16695f4e868 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:43.319059 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c531c3f-0a58-4582-9a5b-f16695f4e868 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:43.337638 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd3843-96c3-4913-a853-7bdc91964be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:43.338460 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd3843-96c3-4913-a853-7bdc91964be5 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:43.338926 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd3843-96c3-4913-a853-7bdc91964be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:43.339368 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd3843-96c3-4913-a853-7bdc91964be5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:43.340443 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cd3843-96c3-4913-a853-7bdc91964be5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:43.358152 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cd3843-96c3-4913-a853-7bdc91964be5 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:43.486811 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 977/1935] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:31:43 2022] POST /identity/v3/projects => generated 383 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:31:43.488158 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d86243-0ce5-4cd2-985d-517939091741 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:43.516091 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d86243-0ce5-4cd2-985d-517939091741 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:43.589863 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574481d-45ec-484c-97b5-020b2baff4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:43.590509 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574481d-45ec-484c-97b5-020b2baff4dd admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:43.590796 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574481d-45ec-484c-97b5-020b2baff4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:43.591270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574481d-45ec-484c-97b5-020b2baff4dd admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:43.592228 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c574481d-45ec-484c-97b5-020b2baff4dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:43.594756 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c574481d-45ec-484c-97b5-020b2baff4dd admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:43.707967 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 959/1936] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:31:43 2022] POST /identity/v3/users => generated 298 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:31:43.711878 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd3843-96c3-4913-a853-7bdc91964be5 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:43.722251 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd3843-96c3-4913-a853-7bdc91964be5 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:43.727089 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff938a-14a4-4748-8446-bae8e502fba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:43.727710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff938a-14a4-4748-8446-bae8e502fba4 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:43.727997 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff938a-14a4-4748-8446-bae8e502fba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:43.728314 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff938a-14a4-4748-8446-bae8e502fba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:43.729050 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff938a-14a4-4748-8446-bae8e502fba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:43.730610 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff938a-14a4-4748-8446-bae8e502fba4 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:43.742187 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 978/1937] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:31:43.746659 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574481d-45ec-484c-97b5-020b2baff4dd admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:43.761053 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574481d-45ec-484c-97b5-020b2baff4dd admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:43.766111 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba530fde-86cd-4f66-9d84-03c14313c99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:43.766887 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba530fde-86cd-4f66-9d84-03c14313c99e admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:43.767385 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba530fde-86cd-4f66-9d84-03c14313c99e admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:43.767736 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba530fde-86cd-4f66-9d84-03c14313c99e admin admin] PATH_INFO: `/v3/projects/3286af1b3a0e4ee9884229f276a9e9a4/users/3192f4e3ba7c46f385fddf9aa2eb82ee/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:43.785005 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba530fde-86cd-4f66-9d84-03c14313c99e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3286af1b3a0e4ee9884229f276a9e9a4, user_id=3192f4e3ba7c46f385fddf9aa2eb82ee, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:43.788082 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba530fde-86cd-4f66-9d84-03c14313c99e admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:43.904581 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:43.918541 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 960/1938] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:31:43 2022] PUT /identity/v3/projects/3286af1b3a0e4ee9884229f276a9e9a4/users/3192f4e3ba7c46f385fddf9aa2eb82ee/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:43.920553 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff938a-14a4-4748-8446-bae8e502fba4 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:43.935655 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff938a-14a4-4748-8446-bae8e502fba4 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:44.028978 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b74ead0-9bac-4f45-872e-82e2599df40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:44.029841 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74ead0-9bac-4f45-872e-82e2599df40b admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:44.030353 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74ead0-9bac-4f45-872e-82e2599df40b admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:44.030936 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74ead0-9bac-4f45-872e-82e2599df40b admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:44.031990 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b74ead0-9bac-4f45-872e-82e2599df40b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:44.034398 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b74ead0-9bac-4f45-872e-82e2599df40b admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:44.049875 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 979/1939] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:31:44.054029 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba530fde-86cd-4f66-9d84-03c14313c99e admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:44.065274 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba530fde-86cd-4f66-9d84-03c14313c99e admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:44.070202 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81810f9d-80bc-4267-b163-f42d50af00bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:44.070893 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81810f9d-80bc-4267-b163-f42d50af00bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:44.071176 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81810f9d-80bc-4267-b163-f42d50af00bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:44.071458 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81810f9d-80bc-4267-b163-f42d50af00bd admin admin] PATH_INFO: `/v3/projects/3286af1b3a0e4ee9884229f276a9e9a4/users/3192f4e3ba7c46f385fddf9aa2eb82ee/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:44.073497 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81810f9d-80bc-4267-b163-f42d50af00bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3286af1b3a0e4ee9884229f276a9e9a4, user_id=3192f4e3ba7c46f385fddf9aa2eb82ee, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:44.076994 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81810f9d-80bc-4267-b163-f42d50af00bd admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:44.090087 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 961/1940] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:31:44 2022] PUT /identity/v3/projects/3286af1b3a0e4ee9884229f276a9e9a4/users/3192f4e3ba7c46f385fddf9aa2eb82ee/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:44.101528 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3593c-28df-408e-a70b-3808b2494fac None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:44.101821 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3593c-28df-408e-a70b-3808b2494fac None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:44.102111 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3593c-28df-408e-a70b-3808b2494fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:44.112289 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:44.159314 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-ccb3593c-28df-408e-a70b-3808b2494fac None None] MFA Rules not processed for user `3192f4e3ba7c46f385fddf9aa2eb82ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:44.160615 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-ccb3593c-28df-408e-a70b-3808b2494fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:44.263915 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 980/1941] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:44 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:44.437309 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81810f9d-80bc-4267-b163-f42d50af00bd admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:44.439800 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-81810f9d-80bc-4267-b163-f42d50af00bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:44.504156 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81810f9d-80bc-4267-b163-f42d50af00bd admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:44.509292 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7992c6-1726-495f-9485-e719c665d8a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:44.510198 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7992c6-1726-495f-9485-e719c665d8a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:44.511070 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7992c6-1726-495f-9485-e719c665d8a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:44.511500 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7992c6-1726-495f-9485-e719c665d8a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:44.598575 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7992c6-1726-495f-9485-e719c665d8a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:44.600315 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7992c6-1726-495f-9485-e719c665d8a2 service cinder] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:44.612803 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 962/1942] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:31:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:31:44.930554 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:45.631747 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:46.306889 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:46.528970 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb3593c-28df-408e-a70b-3808b2494fac None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:46.533391 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-ccb3593c-28df-408e-a70b-3808b2494fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:46.604324 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb3593c-28df-408e-a70b-3808b2494fac None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:46.609699 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204aa0dc-36b2-4506-bce6-286995f4b038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:46.610385 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204aa0dc-36b2-4506-bce6-286995f4b038 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:46.610675 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204aa0dc-36b2-4506-bce6-286995f4b038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:46.610957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204aa0dc-36b2-4506-bce6-286995f4b038 admin admin] PATH_INFO: `/v3/users/11e69daf4f0b40bfb2a18a259b70c8cb` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:46.612273 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204aa0dc-36b2-4506-bce6-286995f4b038 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11e69daf4f0b40bfb2a18a259b70c8cb)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:46.614347 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204aa0dc-36b2-4506-bce6-286995f4b038 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:46.834623 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-204aa0dc-36b2-4506-bce6-286995f4b038 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11e69daf4f0b40bfb2a18a259b70c8cb'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:46.847186 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-204aa0dc-36b2-4506-bce6-286995f4b038 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11e69daf4f0b40bfb2a18a259b70c8cb'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:46.876293 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-204aa0dc-36b2-4506-bce6-286995f4b038 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11e69daf4f0b40bfb2a18a259b70c8cb'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:46.964394 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-204aa0dc-36b2-4506-bce6-286995f4b038 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11e69daf4f0b40bfb2a18a259b70c8cb'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:46.973326 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-204aa0dc-36b2-4506-bce6-286995f4b038 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11e69daf4f0b40bfb2a18a259b70c8cb'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:46.983027 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 981/1943] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:31:46 2022] DELETE /identity/v3/users/11e69daf4f0b40bfb2a18a259b70c8cb => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:46.985390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ccbf53-45b6-4c48-9078-a71323591011 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:46.986128 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ccbf53-45b6-4c48-9078-a71323591011 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:46.986414 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ccbf53-45b6-4c48-9078-a71323591011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:47.008713 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-70ccbf53-45b6-4c48-9078-a71323591011 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:47.009970 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-70ccbf53-45b6-4c48-9078-a71323591011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:47.072650 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 963/1944] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:47.083454 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204aa0dc-36b2-4506-bce6-286995f4b038 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:47.089527 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-204aa0dc-36b2-4506-bce6-286995f4b038 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:47.152778 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204aa0dc-36b2-4506-bce6-286995f4b038 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:47.156591 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c5b995-9620-4176-853b-c0f2bea43052 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:47.157748 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5b995-9620-4176-853b-c0f2bea43052 service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:47.161320 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5b995-9620-4176-853b-c0f2bea43052 service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:47.161320 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5b995-9620-4176-853b-c0f2bea43052 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:47.167712 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c5b995-9620-4176-853b-c0f2bea43052 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:47.169312 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c5b995-9620-4176-853b-c0f2bea43052 service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:47.180167 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 982/1945] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:31:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:31:47.416745 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:47.484688 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:47.630965 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ccbf53-45b6-4c48-9078-a71323591011 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:47.633175 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-70ccbf53-45b6-4c48-9078-a71323591011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:47.689248 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ccbf53-45b6-4c48-9078-a71323591011 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:47.729285 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b60e9d5-4f4a-4af8-b0b9-a7f876e065a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:47.730987 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60e9d5-4f4a-4af8-b0b9-a7f876e065a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:47.730987 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60e9d5-4f4a-4af8-b0b9-a7f876e065a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:47.730987 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60e9d5-4f4a-4af8-b0b9-a7f876e065a6 admin admin] PATH_INFO: `/v3/projects/2f4668b357ae4baf957680eeacf627fe` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:47.732942 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b60e9d5-4f4a-4af8-b0b9-a7f876e065a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f4668b357ae4baf957680eeacf627fe)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:47.735549 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b60e9d5-4f4a-4af8-b0b9-a7f876e065a6 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:47.906394 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-9b60e9d5-4f4a-4af8-b0b9-a7f876e065a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f4668b357ae4baf957680eeacf627fe', 'request_id': 'req-9b60e9d5-4f4a-4af8-b0b9-a7f876e065a6'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:47.917961 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-9b60e9d5-4f4a-4af8-b0b9-a7f876e065a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f4668b357ae4baf957680eeacf627fe', 'request_id': 'req-9b60e9d5-4f4a-4af8-b0b9-a7f876e065a6'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:47.992440 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-9b60e9d5-4f4a-4af8-b0b9-a7f876e065a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f4668b357ae4baf957680eeacf627fe', 'request_id': 'req-9b60e9d5-4f4a-4af8-b0b9-a7f876e065a6'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:48.004697 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-9b60e9d5-4f4a-4af8-b0b9-a7f876e065a6 admin admin] The token cache is being invalidate because project 2f4668b357ae4baf957680eeacf627fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:31:48.004989 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-9b60e9d5-4f4a-4af8-b0b9-a7f876e065a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:48.005622 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-9b60e9d5-4f4a-4af8-b0b9-a7f876e065a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:48.007447 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 964/1946] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:31:47 2022] DELETE /identity/v3/projects/2f4668b357ae4baf957680eeacf627fe => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:48.016047 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c5b995-9620-4176-853b-c0f2bea43052 service neutron] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:48.022489 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c5b995-9620-4176-853b-c0f2bea43052 service neutron] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:48.087642 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3eb06-4c90-428c-b458-52b8d8f46336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:48.088242 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3eb06-4c90-428c-b458-52b8d8f46336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:48.088534 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3eb06-4c90-428c-b458-52b8d8f46336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:48.088818 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3eb06-4c90-428c-b458-52b8d8f46336 admin admin] PATH_INFO: `/v3/users/4e1a532237bb42cb9f287efa0af360a6` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:48.090052 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d3eb06-4c90-428c-b458-52b8d8f46336 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e1a532237bb42cb9f287efa0af360a6)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:48.092043 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d3eb06-4c90-428c-b458-52b8d8f46336 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:48.267787 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:48.280989 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-25d3eb06-4c90-428c-b458-52b8d8f46336 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e1a532237bb42cb9f287efa0af360a6'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:48.281670 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-25d3eb06-4c90-428c-b458-52b8d8f46336 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e1a532237bb42cb9f287efa0af360a6'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:48.293518 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-25d3eb06-4c90-428c-b458-52b8d8f46336 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e1a532237bb42cb9f287efa0af360a6'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:48.403956 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-25d3eb06-4c90-428c-b458-52b8d8f46336 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e1a532237bb42cb9f287efa0af360a6'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:48.412309 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-25d3eb06-4c90-428c-b458-52b8d8f46336 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e1a532237bb42cb9f287efa0af360a6'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:48.418257 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 983/1947] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:31:48 2022] DELETE /identity/v3/users/4e1a532237bb42cb9f287efa0af360a6 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:48.507217 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:48.881421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b60e9d5-4f4a-4af8-b0b9-a7f876e065a6 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:48.883197 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-9b60e9d5-4f4a-4af8-b0b9-a7f876e065a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:48.924042 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b60e9d5-4f4a-4af8-b0b9-a7f876e065a6 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:49.001400 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60393e26-fa5b-4ab7-b7e8-aaa7c28d614e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:49.002289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60393e26-fa5b-4ab7-b7e8-aaa7c28d614e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:49.002743 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60393e26-fa5b-4ab7-b7e8-aaa7c28d614e admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:49.003363 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60393e26-fa5b-4ab7-b7e8-aaa7c28d614e admin admin] PATH_INFO: `/v3/projects/ed2115d8151647cbafc5d265a84dfbc0` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:49.006388 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60393e26-fa5b-4ab7-b7e8-aaa7c28d614e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed2115d8151647cbafc5d265a84dfbc0)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:49.008351 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60393e26-fa5b-4ab7-b7e8-aaa7c28d614e admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:49.126238 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-60393e26-fa5b-4ab7-b7e8-aaa7c28d614e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed2115d8151647cbafc5d265a84dfbc0', 'request_id': 'req-60393e26-fa5b-4ab7-b7e8-aaa7c28d614e'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:49.134122 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-60393e26-fa5b-4ab7-b7e8-aaa7c28d614e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed2115d8151647cbafc5d265a84dfbc0', 'request_id': 'req-60393e26-fa5b-4ab7-b7e8-aaa7c28d614e'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:49.215535 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-60393e26-fa5b-4ab7-b7e8-aaa7c28d614e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed2115d8151647cbafc5d265a84dfbc0', 'request_id': 'req-60393e26-fa5b-4ab7-b7e8-aaa7c28d614e'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:49.227879 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-60393e26-fa5b-4ab7-b7e8-aaa7c28d614e admin admin] The token cache is being invalidate because project ed2115d8151647cbafc5d265a84dfbc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:31:49.228339 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-60393e26-fa5b-4ab7-b7e8-aaa7c28d614e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:49.229081 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-60393e26-fa5b-4ab7-b7e8-aaa7c28d614e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:49.230799 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 965/1948] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:31:48 2022] DELETE /identity/v3/projects/ed2115d8151647cbafc5d265a84dfbc0 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:49.456112 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:49.831740 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:50.722600 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3eb06-4c90-428c-b458-52b8d8f46336 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:50.725735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-25d3eb06-4c90-428c-b458-52b8d8f46336 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:50.791117 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3eb06-4c90-428c-b458-52b8d8f46336 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:50.848545 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfee1c1e-2214-4c80-bad6-4dd8cfa087a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:50.849176 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee1c1e-2214-4c80-bad6-4dd8cfa087a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:50.849561 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee1c1e-2214-4c80-bad6-4dd8cfa087a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:50.849923 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee1c1e-2214-4c80-bad6-4dd8cfa087a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:50.851787 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-cfee1c1e-2214-4c80-bad6-4dd8cfa087a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:50.927039 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfee1c1e-2214-4c80-bad6-4dd8cfa087a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:50.928743 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfee1c1e-2214-4c80-bad6-4dd8cfa087a8 service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:50.939347 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 984/1949] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:31:50 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:31:55.051737 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60393e26-fa5b-4ab7-b7e8-aaa7c28d614e admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:55.055041 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-60393e26-fa5b-4ab7-b7e8-aaa7c28d614e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:55.067860 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:55.119303 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60393e26-fa5b-4ab7-b7e8-aaa7c28d614e admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:55.123994 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d596de72-dedc-45d3-9cc8-6e076b96b321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:55.124713 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596de72-dedc-45d3-9cc8-6e076b96b321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:55.125525 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596de72-dedc-45d3-9cc8-6e076b96b321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:55.125803 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596de72-dedc-45d3-9cc8-6e076b96b321 admin admin] PATH_INFO: `/v3/users/29bd7f872bed4bf280ae07c4bf82753b` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:55.126961 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d596de72-dedc-45d3-9cc8-6e076b96b321 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29bd7f872bed4bf280ae07c4bf82753b)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:55.128939 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d596de72-dedc-45d3-9cc8-6e076b96b321 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:55.297430 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-d596de72-dedc-45d3-9cc8-6e076b96b321 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29bd7f872bed4bf280ae07c4bf82753b'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:55.304259 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-d596de72-dedc-45d3-9cc8-6e076b96b321 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29bd7f872bed4bf280ae07c4bf82753b'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:55.304959 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-d596de72-dedc-45d3-9cc8-6e076b96b321 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29bd7f872bed4bf280ae07c4bf82753b'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:55.317239 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-d596de72-dedc-45d3-9cc8-6e076b96b321 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29bd7f872bed4bf280ae07c4bf82753b'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:55.317885 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-d596de72-dedc-45d3-9cc8-6e076b96b321 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29bd7f872bed4bf280ae07c4bf82753b'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:55.380723 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 966/1950] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:31:55 2022] DELETE /identity/v3/users/29bd7f872bed4bf280ae07c4bf82753b => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:55.385366 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a30a2-b29e-4816-960a-35300df1c576 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:55.385825 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a30a2-b29e-4816-960a-35300df1c576 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:55.385934 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a30a2-b29e-4816-960a-35300df1c576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:55.404751 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-f38a30a2-b29e-4816-960a-35300df1c576 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:55.405946 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-f38a30a2-b29e-4816-960a-35300df1c576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:55.414025 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:55.465197 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 985/1951] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:55.474863 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d596de72-dedc-45d3-9cc8-6e076b96b321 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:55.476300 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-d596de72-dedc-45d3-9cc8-6e076b96b321 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:55.533683 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d596de72-dedc-45d3-9cc8-6e076b96b321 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:55.537304 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39301e69-cfba-425d-ac23-72d444d630c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:55.537958 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39301e69-cfba-425d-ac23-72d444d630c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:55.538243 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39301e69-cfba-425d-ac23-72d444d630c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:55.538550 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39301e69-cfba-425d-ac23-72d444d630c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:55.548024 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39301e69-cfba-425d-ac23-72d444d630c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:55.549482 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39301e69-cfba-425d-ac23-72d444d630c2 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:55.558883 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 967/1952] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:31:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:31:55.980454 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38a30a2-b29e-4816-960a-35300df1c576 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:55.984320 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-f38a30a2-b29e-4816-960a-35300df1c576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:55.998417 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38a30a2-b29e-4816-960a-35300df1c576 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:56.004103 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df95a57c-cc07-44ec-b582-fed637977062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:56.004783 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95a57c-cc07-44ec-b582-fed637977062 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:56.005273 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95a57c-cc07-44ec-b582-fed637977062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:56.005552 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95a57c-cc07-44ec-b582-fed637977062 admin admin] PATH_INFO: `/v3/projects/bb8e2bf04bf24c24b37ddfc80217a84e` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:56.010058 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df95a57c-cc07-44ec-b582-fed637977062 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb8e2bf04bf24c24b37ddfc80217a84e)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:56.010058 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df95a57c-cc07-44ec-b582-fed637977062 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:56.220984 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-df95a57c-cc07-44ec-b582-fed637977062 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb8e2bf04bf24c24b37ddfc80217a84e', 'request_id': 'req-df95a57c-cc07-44ec-b582-fed637977062'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:56.227163 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-df95a57c-cc07-44ec-b582-fed637977062 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb8e2bf04bf24c24b37ddfc80217a84e', 'request_id': 'req-df95a57c-cc07-44ec-b582-fed637977062'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:56.232754 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-df95a57c-cc07-44ec-b582-fed637977062 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb8e2bf04bf24c24b37ddfc80217a84e', 'request_id': 'req-df95a57c-cc07-44ec-b582-fed637977062'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:56.334403 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-df95a57c-cc07-44ec-b582-fed637977062 admin admin] The token cache is being invalidate because project bb8e2bf04bf24c24b37ddfc80217a84e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:31:56.334622 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-df95a57c-cc07-44ec-b582-fed637977062 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:56.335264 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-df95a57c-cc07-44ec-b582-fed637977062 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:56.336889 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 986/1953] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:31:55 2022] DELETE /identity/v3/projects/bb8e2bf04bf24c24b37ddfc80217a84e => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:56.341105 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39301e69-cfba-425d-ac23-72d444d630c2 service neutron] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:56.351364 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39301e69-cfba-425d-ac23-72d444d630c2 service neutron] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:56.385165 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:56.413242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ce27ec-1147-4b33-a7de-64f5ac675760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:56.413925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce27ec-1147-4b33-a7de-64f5ac675760 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:56.414172 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce27ec-1147-4b33-a7de-64f5ac675760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:56.414451 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce27ec-1147-4b33-a7de-64f5ac675760 admin admin] PATH_INFO: `/v3/users/7a6efb71b0e74aaa86c6fb4993970bf8` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:56.415610 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ce27ec-1147-4b33-a7de-64f5ac675760 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a6efb71b0e74aaa86c6fb4993970bf8)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:56.417674 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ce27ec-1147-4b33-a7de-64f5ac675760 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:56.606158 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-c4ce27ec-1147-4b33-a7de-64f5ac675760 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a6efb71b0e74aaa86c6fb4993970bf8'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:56.613729 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-c4ce27ec-1147-4b33-a7de-64f5ac675760 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a6efb71b0e74aaa86c6fb4993970bf8'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:56.614406 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-c4ce27ec-1147-4b33-a7de-64f5ac675760 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a6efb71b0e74aaa86c6fb4993970bf8'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:56.627176 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-c4ce27ec-1147-4b33-a7de-64f5ac675760 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a6efb71b0e74aaa86c6fb4993970bf8'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:56.627825 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-c4ce27ec-1147-4b33-a7de-64f5ac675760 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a6efb71b0e74aaa86c6fb4993970bf8'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:56.701166 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 968/1954] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:31:56 2022] DELETE /identity/v3/users/7a6efb71b0e74aaa86c6fb4993970bf8 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:57.134484 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df95a57c-cc07-44ec-b582-fed637977062 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:57.136420 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-df95a57c-cc07-44ec-b582-fed637977062 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:57.180435 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df95a57c-cc07-44ec-b582-fed637977062 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:57.223072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ce27ec-1147-4b33-a7de-64f5ac675760 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:57.224723 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-c4ce27ec-1147-4b33-a7de-64f5ac675760 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:57.252828 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c00345b-66be-46ec-8277-b5c7c5ffad7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:57.253845 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00345b-66be-46ec-8277-b5c7c5ffad7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:57.254312 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00345b-66be-46ec-8277-b5c7c5ffad7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:57.255085 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00345b-66be-46ec-8277-b5c7c5ffad7f admin admin] PATH_INFO: `/v3/users/a8268ff3ab6248ea924648911dc6a1e6` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:57.258690 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c00345b-66be-46ec-8277-b5c7c5ffad7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8268ff3ab6248ea924648911dc6a1e6)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:57.261135 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c00345b-66be-46ec-8277-b5c7c5ffad7f admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:57.340110 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ce27ec-1147-4b33-a7de-64f5ac675760 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:57.355887 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc53f0-1e12-4889-a702-eab345654bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:57.356624 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc53f0-1e12-4889-a702-eab345654bca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:57.358148 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc53f0-1e12-4889-a702-eab345654bca admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:57.358593 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc53f0-1e12-4889-a702-eab345654bca admin admin] PATH_INFO: `/v3/projects/87ce220cb82e438183b3483fc3917544` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:57.361983 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fc53f0-1e12-4889-a702-eab345654bca admin admin] RBAC: Authorizing `identity:delete_project(project_id=87ce220cb82e438183b3483fc3917544)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:57.375867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fc53f0-1e12-4889-a702-eab345654bca admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:57.571751 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:57.579308 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-b8fc53f0-1e12-4889-a702-eab345654bca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87ce220cb82e438183b3483fc3917544', 'request_id': 'req-b8fc53f0-1e12-4889-a702-eab345654bca'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:57.580744 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-5c00345b-66be-46ec-8277-b5c7c5ffad7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8268ff3ab6248ea924648911dc6a1e6'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:57.581762 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-5c00345b-66be-46ec-8277-b5c7c5ffad7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8268ff3ab6248ea924648911dc6a1e6'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:57.586084 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-b8fc53f0-1e12-4889-a702-eab345654bca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87ce220cb82e438183b3483fc3917544', 'request_id': 'req-b8fc53f0-1e12-4889-a702-eab345654bca'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:57.590168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:57.601103 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-5c00345b-66be-46ec-8277-b5c7c5ffad7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8268ff3ab6248ea924648911dc6a1e6'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:57.672301 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-b8fc53f0-1e12-4889-a702-eab345654bca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87ce220cb82e438183b3483fc3917544', 'request_id': 'req-b8fc53f0-1e12-4889-a702-eab345654bca'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:57.684361 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-b8fc53f0-1e12-4889-a702-eab345654bca admin admin] The token cache is being invalidate because project 87ce220cb82e438183b3483fc3917544 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:31:57.684656 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-b8fc53f0-1e12-4889-a702-eab345654bca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:57.685566 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-b8fc53f0-1e12-4889-a702-eab345654bca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:57.687282 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 969/1955] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:31:57 2022] DELETE /identity/v3/projects/87ce220cb82e438183b3483fc3917544 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:57.692338 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5d353-f4da-4eed-b066-e0910ea90e2f None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:57.692746 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5d353-f4da-4eed-b066-e0910ea90e2f None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:57.693473 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5d353-f4da-4eed-b066-e0910ea90e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:57.715289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-0be5d353-f4da-4eed-b066-e0910ea90e2f None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:57.716749 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-0be5d353-f4da-4eed-b066-e0910ea90e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:57.730763 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-5c00345b-66be-46ec-8277-b5c7c5ffad7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8268ff3ab6248ea924648911dc6a1e6'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:57.742400 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-5c00345b-66be-46ec-8277-b5c7c5ffad7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8268ff3ab6248ea924648911dc6a1e6'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:57.748976 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 987/1956] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:31:57 2022] DELETE /identity/v3/users/a8268ff3ab6248ea924648911dc6a1e6 => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:57.753150 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d1da8-f4ed-4b02-af78-b8f480e92a2f None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:57.753549 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d1da8-f4ed-4b02-af78-b8f480e92a2f None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:57.754112 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d1da8-f4ed-4b02-af78-b8f480e92a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:57.777713 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-be9d1da8-f4ed-4b02-af78-b8f480e92a2f None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:57.779082 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-be9d1da8-f4ed-4b02-af78-b8f480e92a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:57.791721 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 970/1957] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:57.801798 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283aada-ba1e-4a2d-be91-331c1761d482 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:57.802538 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283aada-ba1e-4a2d-be91-331c1761d482 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:57.802538 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283aada-ba1e-4a2d-be91-331c1761d482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:57.826804 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-f283aada-ba1e-4a2d-be91-331c1761d482 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:57.828910 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-f283aada-ba1e-4a2d-be91-331c1761d482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:57.844829 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 988/1958] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:57.854796 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d1da8-f4ed-4b02-af78-b8f480e92a2f None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:57.856390 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-be9d1da8-f4ed-4b02-af78-b8f480e92a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:57.905205 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 971/1959] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:31:57.910056 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f283aada-ba1e-4a2d-be91-331c1761d482 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:57.919767 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f283aada-ba1e-4a2d-be91-331c1761d482 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:57.924501 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd46be0-32f6-4ff0-88de-38e8dbff3135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:57.925066 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd46be0-32f6-4ff0-88de-38e8dbff3135 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:57.925351 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd46be0-32f6-4ff0-88de-38e8dbff3135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:57.925642 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd46be0-32f6-4ff0-88de-38e8dbff3135 admin admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:57.926131 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d1da8-f4ed-4b02-af78-b8f480e92a2f None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:57.926350 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd46be0-32f6-4ff0-88de-38e8dbff3135 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:31:57.926622 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd46be0-32f6-4ff0-88de-38e8dbff3135 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:57.928262 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd46be0-32f6-4ff0-88de-38e8dbff3135 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:57.930292 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1559d03b-f1ec-4f3c-b8da-d0f484d05865 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:57.930924 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1559d03b-f1ec-4f3c-b8da-d0f484d05865 service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:57.931215 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1559d03b-f1ec-4f3c-b8da-d0f484d05865 service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:57.931506 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1559d03b-f1ec-4f3c-b8da-d0f484d05865 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:57.941839 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 972/1960] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:31:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:31:57.946566 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd46be0-32f6-4ff0-88de-38e8dbff3135 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:57.947770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1559d03b-f1ec-4f3c-b8da-d0f484d05865 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:57.949346 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1559d03b-f1ec-4f3c-b8da-d0f484d05865 service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:57.957974 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd46be0-32f6-4ff0-88de-38e8dbff3135 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:57.963408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 989/1961] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:31:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 15:31:57.965735 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426aa4c-a02f-443c-9494-5d1b09c5a555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:57.966997 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426aa4c-a02f-443c-9494-5d1b09c5a555 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:57.967356 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426aa4c-a02f-443c-9494-5d1b09c5a555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:57.967721 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426aa4c-a02f-443c-9494-5d1b09c5a555 admin admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:57.968739 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2426aa4c-a02f-443c-9494-5d1b09c5a555 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:57.971242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2426aa4c-a02f-443c-9494-5d1b09c5a555 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:58.242625 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 973/1962] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:31:57 2022] POST /identity/v3/projects => generated 365 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:31:58.244959 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1559d03b-f1ec-4f3c-b8da-d0f484d05865 service neutron] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:58.255397 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1559d03b-f1ec-4f3c-b8da-d0f484d05865 service neutron] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:58.311834 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9400b124-cadf-49d9-a23c-e3f4bba72adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:58.312992 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400b124-cadf-49d9-a23c-e3f4bba72adf admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:58.313283 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400b124-cadf-49d9-a23c-e3f4bba72adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:58.313564 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400b124-cadf-49d9-a23c-e3f4bba72adf admin admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:58.314233 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9400b124-cadf-49d9-a23c-e3f4bba72adf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:58.316287 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9400b124-cadf-49d9-a23c-e3f4bba72adf admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:58.428661 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 990/1963] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:31:58 2022] POST /identity/v3/users => generated 283 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:31:58.433689 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426aa4c-a02f-443c-9494-5d1b09c5a555 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:58.448981 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426aa4c-a02f-443c-9494-5d1b09c5a555 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:58.459015 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7229a196-d323-414a-b06f-c47553d6bc8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:58.459861 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229a196-d323-414a-b06f-c47553d6bc8e admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:58.460302 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229a196-d323-414a-b06f-c47553d6bc8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:58.460684 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229a196-d323-414a-b06f-c47553d6bc8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:58.461914 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7229a196-d323-414a-b06f-c47553d6bc8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:58.464169 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7229a196-d323-414a-b06f-c47553d6bc8e admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:58.471157 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:58.503894 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 974/1964] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:31:58 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:31:58.509259 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9400b124-cadf-49d9-a23c-e3f4bba72adf admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:58.528286 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9400b124-cadf-49d9-a23c-e3f4bba72adf admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:58.536762 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf97bc7-b361-40b3-8cf5-48e15a4d1e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:58.539985 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf97bc7-b361-40b3-8cf5-48e15a4d1e59 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:58.539985 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf97bc7-b361-40b3-8cf5-48e15a4d1e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:58.539985 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf97bc7-b361-40b3-8cf5-48e15a4d1e59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:58.539985 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf97bc7-b361-40b3-8cf5-48e15a4d1e59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:58.541862 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf97bc7-b361-40b3-8cf5-48e15a4d1e59 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:58.557260 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 991/1965] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:31:58 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:31:58.562986 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7229a196-d323-414a-b06f-c47553d6bc8e admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:58.574180 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7229a196-d323-414a-b06f-c47553d6bc8e admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:58.589077 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb85879-d97b-414d-9ae6-1029b650f440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:58.589795 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb85879-d97b-414d-9ae6-1029b650f440 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:58.590075 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb85879-d97b-414d-9ae6-1029b650f440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:58.590612 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb85879-d97b-414d-9ae6-1029b650f440 admin admin] PATH_INFO: `/v3/projects/d24c6c847bc04921b7da2949dc1222d2/users/01d02b2aecaf4872a947e07c684e6642/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:58.654017 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb85879-d97b-414d-9ae6-1029b650f440 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d24c6c847bc04921b7da2949dc1222d2, user_id=01d02b2aecaf4872a947e07c684e6642, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:58.654017 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb85879-d97b-414d-9ae6-1029b650f440 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:58.738454 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 975/1966] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:31:58 2022] PUT /identity/v3/projects/d24c6c847bc04921b7da2949dc1222d2/users/01d02b2aecaf4872a947e07c684e6642/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:58.754989 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a1af9-d3d3-4e95-af2b-3855584c3ce2 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:58.755282 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a1af9-d3d3-4e95-af2b-3855584c3ce2 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:58.760558 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a1af9-d3d3-4e95-af2b-3855584c3ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:58.790426 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-536a1af9-d3d3-4e95-af2b-3855584c3ce2 None None] MFA Rules not processed for user `01d02b2aecaf4872a947e07c684e6642`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:58.792321 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-536a1af9-d3d3-4e95-af2b-3855584c3ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:58.868366 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 992/1967] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:58.870152 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb85879-d97b-414d-9ae6-1029b650f440 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:58.871467 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536a1af9-d3d3-4e95-af2b-3855584c3ce2 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:58.872462 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-0fb85879-d97b-414d-9ae6-1029b650f440 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:58.916312 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:58.943667 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536a1af9-d3d3-4e95-af2b-3855584c3ce2 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:58.952165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d663106-70ff-41a4-b69d-f06970dd2a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:58.953387 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d663106-70ff-41a4-b69d-f06970dd2a3b admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:58.954047 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d663106-70ff-41a4-b69d-f06970dd2a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:58.954790 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d663106-70ff-41a4-b69d-f06970dd2a3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:58.955566 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d663106-70ff-41a4-b69d-f06970dd2a3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:58.958179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d663106-70ff-41a4-b69d-f06970dd2a3b admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:58.962031 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb85879-d97b-414d-9ae6-1029b650f440 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:58.966783 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd53ae8-ad33-4636-98b6-aa18e529e069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:58.967367 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd53ae8-ad33-4636-98b6-aa18e529e069 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:58.967747 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd53ae8-ad33-4636-98b6-aa18e529e069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:58.968058 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd53ae8-ad33-4636-98b6-aa18e529e069 admin admin] PATH_INFO: `/v3/projects/43cb2fa3e8a1460f9a68979fb8e9299b` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:58.969875 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd53ae8-ad33-4636-98b6-aa18e529e069 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43cb2fa3e8a1460f9a68979fb8e9299b)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:58.971842 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd53ae8-ad33-4636-98b6-aa18e529e069 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:59.056381 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 993/1968] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:31:58 2022] POST /identity/v3/projects => generated 365 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:31:59.060394 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d663106-70ff-41a4-b69d-f06970dd2a3b admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:59.129119 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d663106-70ff-41a4-b69d-f06970dd2a3b admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:59.131119 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-4bd53ae8-ad33-4636-98b6-aa18e529e069 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43cb2fa3e8a1460f9a68979fb8e9299b', 'request_id': 'req-4bd53ae8-ad33-4636-98b6-aa18e529e069'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:59.137506 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-4bd53ae8-ad33-4636-98b6-aa18e529e069 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43cb2fa3e8a1460f9a68979fb8e9299b', 'request_id': 'req-4bd53ae8-ad33-4636-98b6-aa18e529e069'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:59.191895 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d19858-bdf2-45a8-b95f-5088775d0c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:59.192571 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d19858-bdf2-45a8-b95f-5088775d0c11 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:59.192867 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d19858-bdf2-45a8-b95f-5088775d0c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:59.193140 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-4bd53ae8-ad33-4636-98b6-aa18e529e069 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43cb2fa3e8a1460f9a68979fb8e9299b', 'request_id': 'req-4bd53ae8-ad33-4636-98b6-aa18e529e069'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:59.193259 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d19858-bdf2-45a8-b95f-5088775d0c11 admin admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:59.193826 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d19858-bdf2-45a8-b95f-5088775d0c11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:59.196507 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d19858-bdf2-45a8-b95f-5088775d0c11 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:59.210033 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-4bd53ae8-ad33-4636-98b6-aa18e529e069 admin admin] The token cache is being invalidate because project 43cb2fa3e8a1460f9a68979fb8e9299b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:31:59.216877 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-4bd53ae8-ad33-4636-98b6-aa18e529e069 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:59.216877 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-4bd53ae8-ad33-4636-98b6-aa18e529e069 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:59.216877 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 976/1969] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:31:58 2022] DELETE /identity/v3/projects/43cb2fa3e8a1460f9a68979fb8e9299b => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:31:59.217264 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd53ae8-ad33-4636-98b6-aa18e529e069 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:59.219192 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-4bd53ae8-ad33-4636-98b6-aa18e529e069 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:59.232498 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd53ae8-ad33-4636-98b6-aa18e529e069 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:59.239460 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab22cf3-2a05-46fe-aa24-a11a11f46ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:59.239460 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab22cf3-2a05-46fe-aa24-a11a11f46ec7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:59.239460 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab22cf3-2a05-46fe-aa24-a11a11f46ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:59.239460 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab22cf3-2a05-46fe-aa24-a11a11f46ec7 admin admin] PATH_INFO: `/v3/users/3192f4e3ba7c46f385fddf9aa2eb82ee` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:59.241137 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab22cf3-2a05-46fe-aa24-a11a11f46ec7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3192f4e3ba7c46f385fddf9aa2eb82ee)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:59.243160 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab22cf3-2a05-46fe-aa24-a11a11f46ec7 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:59.284105 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 994/1970] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:31:59 2022] POST /identity/v3/users => generated 289 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:31:59.287949 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d19858-bdf2-45a8-b95f-5088775d0c11 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:59.290302 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-69d19858-bdf2-45a8-b95f-5088775d0c11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:59.305674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:59.309924 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d19858-bdf2-45a8-b95f-5088775d0c11 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:59.316777 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc08fa-6dec-446e-b809-9793eedd126a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:59.319914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc08fa-6dec-446e-b809-9793eedd126a admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:59.319914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc08fa-6dec-446e-b809-9793eedd126a admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:59.319914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc08fa-6dec-446e-b809-9793eedd126a admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:59.319914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bc08fa-6dec-446e-b809-9793eedd126a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:59.320795 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bc08fa-6dec-446e-b809-9793eedd126a admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:59.332976 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 995/1971] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:31:59 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 15:31:59.337620 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc08fa-6dec-446e-b809-9793eedd126a admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:59.354658 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc08fa-6dec-446e-b809-9793eedd126a admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:59.368527 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc5dce-05a9-4d5e-adad-bf1b1d499961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:59.369443 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc5dce-05a9-4d5e-adad-bf1b1d499961 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:59.369921 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc5dce-05a9-4d5e-adad-bf1b1d499961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:59.370306 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc5dce-05a9-4d5e-adad-bf1b1d499961 admin admin] PATH_INFO: `/v3/projects/1140834102cf47a88a14145e62f8e14c/users/9203e33c023c401fb4d6cc2d686c0175/roles/7c88c815db854954b516ab97cd471647` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:59.398458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cc5dce-05a9-4d5e-adad-bf1b1d499961 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1140834102cf47a88a14145e62f8e14c, user_id=9203e33c023c401fb4d6cc2d686c0175, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:59.398458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cc5dce-05a9-4d5e-adad-bf1b1d499961 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:59.429580 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-1ab22cf3-2a05-46fe-aa24-a11a11f46ec7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3192f4e3ba7c46f385fddf9aa2eb82ee'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:59.437231 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-1ab22cf3-2a05-46fe-aa24-a11a11f46ec7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3192f4e3ba7c46f385fddf9aa2eb82ee'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:59.437967 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-1ab22cf3-2a05-46fe-aa24-a11a11f46ec7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3192f4e3ba7c46f385fddf9aa2eb82ee'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:59.450983 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-1ab22cf3-2a05-46fe-aa24-a11a11f46ec7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3192f4e3ba7c46f385fddf9aa2eb82ee'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:59.451824 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-1ab22cf3-2a05-46fe-aa24-a11a11f46ec7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3192f4e3ba7c46f385fddf9aa2eb82ee'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:31:59.473024 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 996/1972] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:31:59 2022] PUT /identity/v3/projects/1140834102cf47a88a14145e62f8e14c/users/9203e33c023c401fb4d6cc2d686c0175/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:31:59.476977 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc5dce-05a9-4d5e-adad-bf1b1d499961 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:59.478696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-53cc5dce-05a9-4d5e-adad-bf1b1d499961 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:59.546566 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc5dce-05a9-4d5e-adad-bf1b1d499961 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:59.547005 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 977/1973] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:31:59 2022] DELETE /identity/v3/users/3192f4e3ba7c46f385fddf9aa2eb82ee => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:31:59.605646 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7044b17d-81bf-4ef0-bd63-aaed15a9281c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:59.606284 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7044b17d-81bf-4ef0-bd63-aaed15a9281c admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:59.606575 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7044b17d-81bf-4ef0-bd63-aaed15a9281c admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:59.606858 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7044b17d-81bf-4ef0-bd63-aaed15a9281c admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:59.607635 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7044b17d-81bf-4ef0-bd63-aaed15a9281c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:59.609862 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7044b17d-81bf-4ef0-bd63-aaed15a9281c admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:59.623451 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 997/1974] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:31:59 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 15:31:59.626648 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab22cf3-2a05-46fe-aa24-a11a11f46ec7 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:59.635751 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab22cf3-2a05-46fe-aa24-a11a11f46ec7 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:59.640582 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbceb02-5ecf-4d6c-a16c-9a3245ec14cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:59.641293 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbceb02-5ecf-4d6c-a16c-9a3245ec14cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:59.641717 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbceb02-5ecf-4d6c-a16c-9a3245ec14cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:59.642002 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbceb02-5ecf-4d6c-a16c-9a3245ec14cf admin admin] PATH_INFO: `/v3/projects/1140834102cf47a88a14145e62f8e14c/users/9203e33c023c401fb4d6cc2d686c0175/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:59.644395 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbceb02-5ecf-4d6c-a16c-9a3245ec14cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1140834102cf47a88a14145e62f8e14c, user_id=9203e33c023c401fb4d6cc2d686c0175, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:59.649346 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbceb02-5ecf-4d6c-a16c-9a3245ec14cf admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:59.657742 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 978/1975] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:31:59 2022] PUT /identity/v3/projects/1140834102cf47a88a14145e62f8e14c/users/9203e33c023c401fb4d6cc2d686c0175/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:31:59.668132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df35b1d-bd00-4a7c-9cd8-b2fb94cdf178 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:59.668531 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df35b1d-bd00-4a7c-9cd8-b2fb94cdf178 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:59.668913 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df35b1d-bd00-4a7c-9cd8-b2fb94cdf178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:59.701445 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-0df35b1d-bd00-4a7c-9cd8-b2fb94cdf178 None None] MFA Rules not processed for user `9203e33c023c401fb4d6cc2d686c0175`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:31:59.702687 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-0df35b1d-bd00-4a7c-9cd8-b2fb94cdf178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:59.774740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:59.783399 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 998/1976] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:31:59 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:31:59.800048 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbceb02-5ecf-4d6c-a16c-9a3245ec14cf admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:59.802497 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-2fbceb02-5ecf-4d6c-a16c-9a3245ec14cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:31:59.809319 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:31:59.864842 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbceb02-5ecf-4d6c-a16c-9a3245ec14cf admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:31:59.868886 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1aac5-6ddc-4f63-bf1f-aa518f0d7e5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:31:59.869526 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1aac5-6ddc-4f63-bf1f-aa518f0d7e5b service cinder] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:31:59.869816 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1aac5-6ddc-4f63-bf1f-aa518f0d7e5b service cinder] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:31:59.870250 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1aac5-6ddc-4f63-bf1f-aa518f0d7e5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:31:59.880405 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d1aac5-6ddc-4f63-bf1f-aa518f0d7e5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:31:59.881849 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d1aac5-6ddc-4f63-bf1f-aa518f0d7e5b service cinder] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:31:59.891927 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 979/1977] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:31:59 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:31:59.947622 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df35b1d-bd00-4a7c-9cd8-b2fb94cdf178 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:31:59.954001 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-0df35b1d-bd00-4a7c-9cd8-b2fb94cdf178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:00.025382 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df35b1d-bd00-4a7c-9cd8-b2fb94cdf178 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:00.030444 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa26cf-6226-48c4-99ff-34e1b6ae30dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:00.034416 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa26cf-6226-48c4-99ff-34e1b6ae30dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:00.034416 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa26cf-6226-48c4-99ff-34e1b6ae30dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:00.034416 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa26cf-6226-48c4-99ff-34e1b6ae30dd admin admin] PATH_INFO: `/v3/projects/3286af1b3a0e4ee9884229f276a9e9a4` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:00.034416 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caa26cf-6226-48c4-99ff-34e1b6ae30dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3286af1b3a0e4ee9884229f276a9e9a4)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:00.035877 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caa26cf-6226-48c4-99ff-34e1b6ae30dd admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:00.102531 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1aac5-6ddc-4f63-bf1f-aa518f0d7e5b service cinder] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:00.112221 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1aac5-6ddc-4f63-bf1f-aa518f0d7e5b service cinder] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:00.116187 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8863b297-6f37-4d25-92f7-e6f0012d32a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:00.116800 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863b297-6f37-4d25-92f7-e6f0012d32a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:00.117093 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863b297-6f37-4d25-92f7-e6f0012d32a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:00.117375 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863b297-6f37-4d25-92f7-e6f0012d32a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:00.119134 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-8863b297-6f37-4d25-92f7-e6f0012d32a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:00.151888 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-6caa26cf-6226-48c4-99ff-34e1b6ae30dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3286af1b3a0e4ee9884229f276a9e9a4', 'request_id': 'req-6caa26cf-6226-48c4-99ff-34e1b6ae30dd'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:00.158912 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-6caa26cf-6226-48c4-99ff-34e1b6ae30dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3286af1b3a0e4ee9884229f276a9e9a4', 'request_id': 'req-6caa26cf-6226-48c4-99ff-34e1b6ae30dd'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:00.165303 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-6caa26cf-6226-48c4-99ff-34e1b6ae30dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3286af1b3a0e4ee9884229f276a9e9a4', 'request_id': 'req-6caa26cf-6226-48c4-99ff-34e1b6ae30dd'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:00.182372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8863b297-6f37-4d25-92f7-e6f0012d32a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:00.183880 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8863b297-6f37-4d25-92f7-e6f0012d32a6 service cinder] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:00.193533 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 980/1978] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:32:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 15:32:00.226106 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-6caa26cf-6226-48c4-99ff-34e1b6ae30dd admin admin] The token cache is being invalidate because project 3286af1b3a0e4ee9884229f276a9e9a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:32:00.226424 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-6caa26cf-6226-48c4-99ff-34e1b6ae30dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:00.227063 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-6caa26cf-6226-48c4-99ff-34e1b6ae30dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:00.228658 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 999/1979] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:31:59 2022] DELETE /identity/v3/projects/3286af1b3a0e4ee9884229f276a9e9a4 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:00.235602 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b40b23-98f7-4df0-a20a-22a0f106d57c None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:00.236080 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b40b23-98f7-4df0-a20a-22a0f106d57c None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:00.236567 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b40b23-98f7-4df0-a20a-22a0f106d57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:00.256091 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-e7b40b23-98f7-4df0-a20a-22a0f106d57c None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:00.257241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-e7b40b23-98f7-4df0-a20a-22a0f106d57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:00.318028 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 981/1980] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:00.326608 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c108ebb-0e5d-41ff-bfad-4536c5dd2e52 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:00.327171 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c108ebb-0e5d-41ff-bfad-4536c5dd2e52 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:00.327272 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c108ebb-0e5d-41ff-bfad-4536c5dd2e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:00.349553 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-9c108ebb-0e5d-41ff-bfad-4536c5dd2e52 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:00.351586 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-9c108ebb-0e5d-41ff-bfad-4536c5dd2e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:00.361052 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1000/1981] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:00.365305 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b40b23-98f7-4df0-a20a-22a0f106d57c None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:00.376512 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b40b23-98f7-4df0-a20a-22a0f106d57c None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:00.381375 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5933a53f-9b84-4fca-aee1-47d03c968f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:00.381996 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933a53f-9b84-4fca-aee1-47d03c968f41 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:00.382314 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933a53f-9b84-4fca-aee1-47d03c968f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:00.382593 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933a53f-9b84-4fca-aee1-47d03c968f41 admin admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:00.383536 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5933a53f-9b84-4fca-aee1-47d03c968f41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:32:00.383854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5933a53f-9b84-4fca-aee1-47d03c968f41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:00.385535 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5933a53f-9b84-4fca-aee1-47d03c968f41 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:00.397900 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 982/1982] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:32:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:32:00.402441 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c108ebb-0e5d-41ff-bfad-4536c5dd2e52 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:00.412456 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c108ebb-0e5d-41ff-bfad-4536c5dd2e52 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:00.417065 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb75636-6b71-4991-b96e-f1ea7946ceaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:00.417719 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb75636-6b71-4991-b96e-f1ea7946ceaa admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:00.418008 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb75636-6b71-4991-b96e-f1ea7946ceaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:00.418289 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb75636-6b71-4991-b96e-f1ea7946ceaa admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:00.418953 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb75636-6b71-4991-b96e-f1ea7946ceaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:00.420891 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb75636-6b71-4991-b96e-f1ea7946ceaa admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:00.425379 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:00.500124 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1001/1983] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:32:00 2022] POST /identity/v3/projects => generated 371 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:32:00.501547 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5933a53f-9b84-4fca-aee1-47d03c968f41 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:00.515492 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5933a53f-9b84-4fca-aee1-47d03c968f41 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:00.549065 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:00.575953 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb89e2e0-c314-45c1-a031-027b37b9adca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:00.576612 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89e2e0-c314-45c1-a031-027b37b9adca admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:00.576903 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89e2e0-c314-45c1-a031-027b37b9adca admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:00.577187 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89e2e0-c314-45c1-a031-027b37b9adca admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:00.578522 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb89e2e0-c314-45c1-a031-027b37b9adca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:00.580481 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb89e2e0-c314-45c1-a031-027b37b9adca admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:00.649695 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 983/1984] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:32:00 2022] POST /identity/v3/users => generated 286 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:32:00.653127 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb75636-6b71-4991-b96e-f1ea7946ceaa admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:00.663684 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb75636-6b71-4991-b96e-f1ea7946ceaa admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:00.668462 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd46433-6a18-45c2-b389-0525abc77744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:00.669088 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd46433-6a18-45c2-b389-0525abc77744 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:00.669376 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd46433-6a18-45c2-b389-0525abc77744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:00.669664 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd46433-6a18-45c2-b389-0525abc77744 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:00.670410 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd46433-6a18-45c2-b389-0525abc77744 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:00.672052 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd46433-6a18-45c2-b389-0525abc77744 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:00.682395 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1002/1985] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:32:00.686484 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb89e2e0-c314-45c1-a031-027b37b9adca admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:00.698850 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb89e2e0-c314-45c1-a031-027b37b9adca admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:00.704390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ee8ba6-c0e3-430e-b7b9-115bca16634f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:00.704996 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee8ba6-c0e3-430e-b7b9-115bca16634f admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:00.705281 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee8ba6-c0e3-430e-b7b9-115bca16634f admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:00.705566 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee8ba6-c0e3-430e-b7b9-115bca16634f admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:00.706348 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ee8ba6-c0e3-430e-b7b9-115bca16634f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:00.707956 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ee8ba6-c0e3-430e-b7b9-115bca16634f admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:00.720200 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 984/1986] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:32:00.724298 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd46433-6a18-45c2-b389-0525abc77744 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:00.734998 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd46433-6a18-45c2-b389-0525abc77744 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:00.739884 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8308b517-9d52-43c4-81bb-e79552ebddc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:00.740877 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8308b517-9d52-43c4-81bb-e79552ebddc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:00.741164 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8308b517-9d52-43c4-81bb-e79552ebddc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:00.741443 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8308b517-9d52-43c4-81bb-e79552ebddc7 admin admin] PATH_INFO: `/v3/projects/dd21b66440474a94bb01f2945c569033/users/348dc228c6324c6896fcc69dd7e6bcb5/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:00.756829 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8308b517-9d52-43c4-81bb-e79552ebddc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd21b66440474a94bb01f2945c569033, user_id=348dc228c6324c6896fcc69dd7e6bcb5, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:00.759416 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8308b517-9d52-43c4-81bb-e79552ebddc7 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:00.849166 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1003/1987] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:32:00 2022] PUT /identity/v3/projects/dd21b66440474a94bb01f2945c569033/users/348dc228c6324c6896fcc69dd7e6bcb5/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:32:00.860634 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e4a83-14b7-4881-ae70-02d1c2e93d69 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:00.860927 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e4a83-14b7-4881-ae70-02d1c2e93d69 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:00.861210 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e4a83-14b7-4881-ae70-02d1c2e93d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:00.890061 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-3f5e4a83-14b7-4881-ae70-02d1c2e93d69 None None] MFA Rules not processed for user `348dc228c6324c6896fcc69dd7e6bcb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:00.891817 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-3f5e4a83-14b7-4881-ae70-02d1c2e93d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:00.958127 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 985/1988] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:00.969736 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8308b517-9d52-43c4-81bb-e79552ebddc7 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:00.971448 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-8308b517-9d52-43c4-81bb-e79552ebddc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:01.029889 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8308b517-9d52-43c4-81bb-e79552ebddc7 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:01.033572 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3937ce8c-ca9a-4b97-b262-adfe84c16964 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:01.034195 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937ce8c-ca9a-4b97-b262-adfe84c16964 service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:01.034480 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937ce8c-ca9a-4b97-b262-adfe84c16964 service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:01.034762 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937ce8c-ca9a-4b97-b262-adfe84c16964 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:01.043426 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3937ce8c-ca9a-4b97-b262-adfe84c16964 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:01.044933 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3937ce8c-ca9a-4b97-b262-adfe84c16964 service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:01.054370 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1004/1989] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:32:00 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:32:01.076144 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e4a83-14b7-4881-ae70-02d1c2e93d69 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:01.137967 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e4a83-14b7-4881-ae70-02d1c2e93d69 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:01.142660 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cfdc36-b099-4876-9821-76b2ae3a81df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:01.143295 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cfdc36-b099-4876-9821-76b2ae3a81df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:01.143580 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cfdc36-b099-4876-9821-76b2ae3a81df admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:01.143860 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cfdc36-b099-4876-9821-76b2ae3a81df admin admin] PATH_INFO: `/v3/users/348dc228c6324c6896fcc69dd7e6bcb5` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:01.145063 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cfdc36-b099-4876-9821-76b2ae3a81df admin admin] RBAC: Authorizing `identity:delete_user(user_id=348dc228c6324c6896fcc69dd7e6bcb5)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:01.147013 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cfdc36-b099-4876-9821-76b2ae3a81df admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:01.339858 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:01.342331 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-32cfdc36-b099-4876-9821-76b2ae3a81df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '348dc228c6324c6896fcc69dd7e6bcb5'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:01.349679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-32cfdc36-b099-4876-9821-76b2ae3a81df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '348dc228c6324c6896fcc69dd7e6bcb5'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:01.350400 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-32cfdc36-b099-4876-9821-76b2ae3a81df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '348dc228c6324c6896fcc69dd7e6bcb5'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:01.363087 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-32cfdc36-b099-4876-9821-76b2ae3a81df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '348dc228c6324c6896fcc69dd7e6bcb5'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:01.363869 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-32cfdc36-b099-4876-9821-76b2ae3a81df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '348dc228c6324c6896fcc69dd7e6bcb5'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:01.442907 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 986/1990] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:32:01 2022] DELETE /identity/v3/users/348dc228c6324c6896fcc69dd7e6bcb5 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:01.447371 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7980e4f-8593-4b0f-971d-bc689dd5985b None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:01.447846 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7980e4f-8593-4b0f-971d-bc689dd5985b None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:01.447934 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7980e4f-8593-4b0f-971d-bc689dd5985b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:01.467362 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-f7980e4f-8593-4b0f-971d-bc689dd5985b None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:01.469867 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-f7980e4f-8593-4b0f-971d-bc689dd5985b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:01.527064 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1005/1991] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:01.537210 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cfdc36-b099-4876-9821-76b2ae3a81df admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:01.538643 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-32cfdc36-b099-4876-9821-76b2ae3a81df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:01.599603 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cfdc36-b099-4876-9821-76b2ae3a81df admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:01.603338 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23c8b5-92e1-44bf-b639-76bc844102e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:01.604047 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23c8b5-92e1-44bf-b639-76bc844102e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:01.604479 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23c8b5-92e1-44bf-b639-76bc844102e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:01.604821 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23c8b5-92e1-44bf-b639-76bc844102e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:01.614647 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c23c8b5-92e1-44bf-b639-76bc844102e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:01.616365 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c23c8b5-92e1-44bf-b639-76bc844102e6 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:01.626372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 987/1992] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:32:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:32:02.090187 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7980e4f-8593-4b0f-971d-bc689dd5985b None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:02.092124 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-f7980e4f-8593-4b0f-971d-bc689dd5985b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:02.108682 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7980e4f-8593-4b0f-971d-bc689dd5985b None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:02.114565 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1016aea9-dd22-45c8-8822-d402d2493dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:02.115369 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016aea9-dd22-45c8-8822-d402d2493dc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:02.116013 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016aea9-dd22-45c8-8822-d402d2493dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:02.116398 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016aea9-dd22-45c8-8822-d402d2493dc1 admin admin] PATH_INFO: `/v3/projects/dd21b66440474a94bb01f2945c569033` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:02.117509 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1016aea9-dd22-45c8-8822-d402d2493dc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd21b66440474a94bb01f2945c569033)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:02.119367 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1016aea9-dd22-45c8-8822-d402d2493dc1 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:02.266503 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:02.268960 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-1016aea9-dd22-45c8-8822-d402d2493dc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd21b66440474a94bb01f2945c569033', 'request_id': 'req-1016aea9-dd22-45c8-8822-d402d2493dc1'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:02.281788 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-1016aea9-dd22-45c8-8822-d402d2493dc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd21b66440474a94bb01f2945c569033', 'request_id': 'req-1016aea9-dd22-45c8-8822-d402d2493dc1'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:02.288002 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-1016aea9-dd22-45c8-8822-d402d2493dc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd21b66440474a94bb01f2945c569033', 'request_id': 'req-1016aea9-dd22-45c8-8822-d402d2493dc1'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:02.415661 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-1016aea9-dd22-45c8-8822-d402d2493dc1 admin admin] The token cache is being invalidate because project dd21b66440474a94bb01f2945c569033 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:32:02.416933 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-1016aea9-dd22-45c8-8822-d402d2493dc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:02.416933 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-1016aea9-dd22-45c8-8822-d402d2493dc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:02.418376 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1006/1993] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:32:02 2022] DELETE /identity/v3/projects/dd21b66440474a94bb01f2945c569033 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:02.423370 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f37a47-0ac4-4cae-ad86-20e01fb97dd1 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:02.423662 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f37a47-0ac4-4cae-ad86-20e01fb97dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:02.423942 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f37a47-0ac4-4cae-ad86-20e01fb97dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:02.445001 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-a5f37a47-0ac4-4cae-ad86-20e01fb97dd1 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:02.446294 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-a5f37a47-0ac4-4cae-ad86-20e01fb97dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:02.508662 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 988/1994] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:02.517460 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e752682-a289-4e9f-aff5-7a91d73167fe None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:02.518025 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e752682-a289-4e9f-aff5-7a91d73167fe None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:02.518025 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e752682-a289-4e9f-aff5-7a91d73167fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:02.542079 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-8e752682-a289-4e9f-aff5-7a91d73167fe None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:02.543348 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-8e752682-a289-4e9f-aff5-7a91d73167fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:02.554187 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1007/1995] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:02.558242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f37a47-0ac4-4cae-ad86-20e01fb97dd1 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:02.569714 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f37a47-0ac4-4cae-ad86-20e01fb97dd1 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:02.574931 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ad7d8-967a-4f4f-a02c-be559a2bc5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:02.575569 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ad7d8-967a-4f4f-a02c-be559a2bc5ac admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:02.575849 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ad7d8-967a-4f4f-a02c-be559a2bc5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:02.576174 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ad7d8-967a-4f4f-a02c-be559a2bc5ac admin admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:02.576873 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236ad7d8-967a-4f4f-a02c-be559a2bc5ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:32:02.577317 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236ad7d8-967a-4f4f-a02c-be559a2bc5ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:02.580402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236ad7d8-967a-4f4f-a02c-be559a2bc5ac admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:02.594751 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 989/1996] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:32:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:32:02.599403 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e752682-a289-4e9f-aff5-7a91d73167fe None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:02.610639 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e752682-a289-4e9f-aff5-7a91d73167fe None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:02.620351 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce31137-0f53-4c06-bd7e-329ddb572055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:02.621282 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce31137-0f53-4c06-bd7e-329ddb572055 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:02.621570 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce31137-0f53-4c06-bd7e-329ddb572055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:02.621850 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce31137-0f53-4c06-bd7e-329ddb572055 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:02.627128 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce31137-0f53-4c06-bd7e-329ddb572055 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:02.627128 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce31137-0f53-4c06-bd7e-329ddb572055 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:02.699484 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1008/1997] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:32:02 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:32:02.700540 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ad7d8-967a-4f4f-a02c-be559a2bc5ac admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:02.710576 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ad7d8-967a-4f4f-a02c-be559a2bc5ac admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:02.758571 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:02.768133 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78822747-7d21-4fb0-883a-1d977d34197e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:02.768752 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78822747-7d21-4fb0-883a-1d977d34197e admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:02.769037 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78822747-7d21-4fb0-883a-1d977d34197e admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:02.769313 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78822747-7d21-4fb0-883a-1d977d34197e admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:02.769985 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78822747-7d21-4fb0-883a-1d977d34197e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:02.771876 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78822747-7d21-4fb0-883a-1d977d34197e admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:02.848987 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 990/1998] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:32:02 2022] POST /identity/v3/users => generated 287 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:32:02.853283 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce31137-0f53-4c06-bd7e-329ddb572055 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:02.864238 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce31137-0f53-4c06-bd7e-329ddb572055 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:02.869167 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da94b36-db50-43a4-a600-b7934793a6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:02.873021 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da94b36-db50-43a4-a600-b7934793a6e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:02.873021 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da94b36-db50-43a4-a600-b7934793a6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:02.873021 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da94b36-db50-43a4-a600-b7934793a6e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:02.873021 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da94b36-db50-43a4-a600-b7934793a6e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:02.873021 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da94b36-db50-43a4-a600-b7934793a6e1 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:02.884616 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1009/1999] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:32:02.892885 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78822747-7d21-4fb0-883a-1d977d34197e admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:02.902946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78822747-7d21-4fb0-883a-1d977d34197e admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:02.908025 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10f176-a948-42fe-aa6d-ed9af85bf267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:02.908666 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10f176-a948-42fe-aa6d-ed9af85bf267 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:02.908951 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10f176-a948-42fe-aa6d-ed9af85bf267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:02.909230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10f176-a948-42fe-aa6d-ed9af85bf267 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:02.909981 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af10f176-a948-42fe-aa6d-ed9af85bf267 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:02.911551 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af10f176-a948-42fe-aa6d-ed9af85bf267 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:02.922873 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 991/2000] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:32:02.927012 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da94b36-db50-43a4-a600-b7934793a6e1 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:02.937005 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da94b36-db50-43a4-a600-b7934793a6e1 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:02.941680 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f9632b-b097-4ad9-9b1c-5303ab886ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:02.942359 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f9632b-b097-4ad9-9b1c-5303ab886ea0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:02.942635 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f9632b-b097-4ad9-9b1c-5303ab886ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:02.943183 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f9632b-b097-4ad9-9b1c-5303ab886ea0 admin admin] PATH_INFO: `/v3/projects/e15350092bd2477dbf952992e58d1bda/users/aaa1959b3e0a47359653f9e371361149/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:02.958725 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f9632b-b097-4ad9-9b1c-5303ab886ea0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e15350092bd2477dbf952992e58d1bda, user_id=aaa1959b3e0a47359653f9e371361149, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:02.961500 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f9632b-b097-4ad9-9b1c-5303ab886ea0 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:03.032914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1010/2001] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:32:02 2022] PUT /identity/v3/projects/e15350092bd2477dbf952992e58d1bda/users/aaa1959b3e0a47359653f9e371361149/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:03.043380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a3732-d4bb-4bf5-aaa4-e3277ca42020 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:03.043672 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a3732-d4bb-4bf5-aaa4-e3277ca42020 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:03.043948 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a3732-d4bb-4bf5-aaa4-e3277ca42020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:03.073860 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-601a3732-d4bb-4bf5-aaa4-e3277ca42020 None None] MFA Rules not processed for user `aaa1959b3e0a47359653f9e371361149`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:03.075100 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-601a3732-d4bb-4bf5-aaa4-e3277ca42020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:03.143239 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 992/2002] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:03.148108 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f9632b-b097-4ad9-9b1c-5303ab886ea0 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:03.213383 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f9632b-b097-4ad9-9b1c-5303ab886ea0 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:03.218890 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e280452e-0650-462b-aa84-6938aa708861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:03.219541 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280452e-0650-462b-aa84-6938aa708861 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:03.219826 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280452e-0650-462b-aa84-6938aa708861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:03.220131 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280452e-0650-462b-aa84-6938aa708861 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:03.220797 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e280452e-0650-462b-aa84-6938aa708861 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:03.222703 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e280452e-0650-462b-aa84-6938aa708861 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:03.370993 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1011/2003] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:32:03 2022] POST /identity/v3/projects => generated 373 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:32:03.379323 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a3732-d4bb-4bf5-aaa4-e3277ca42020 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:03.390764 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a3732-d4bb-4bf5-aaa4-e3277ca42020 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:03.448494 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c74d69e-7974-4c65-a1ec-fc7860d7d07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:03.449112 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74d69e-7974-4c65-a1ec-fc7860d7d07e admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:03.449397 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74d69e-7974-4c65-a1ec-fc7860d7d07e admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:03.449675 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74d69e-7974-4c65-a1ec-fc7860d7d07e admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:03.450343 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c74d69e-7974-4c65-a1ec-fc7860d7d07e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:03.452804 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c74d69e-7974-4c65-a1ec-fc7860d7d07e admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:03.586154 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 993/2004] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:32:03 2022] POST /identity/v3/users => generated 293 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:32:03.590180 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e280452e-0650-462b-aa84-6938aa708861 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:03.601395 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e280452e-0650-462b-aa84-6938aa708861 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:03.606281 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8956c7-3f6f-4be9-a3ed-21cc2995e4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:03.606999 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8956c7-3f6f-4be9-a3ed-21cc2995e4ba admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:03.607399 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8956c7-3f6f-4be9-a3ed-21cc2995e4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:03.607778 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8956c7-3f6f-4be9-a3ed-21cc2995e4ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:03.608665 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8956c7-3f6f-4be9-a3ed-21cc2995e4ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:03.610260 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8956c7-3f6f-4be9-a3ed-21cc2995e4ba admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:03.621429 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1012/2005] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:32:03.626163 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c74d69e-7974-4c65-a1ec-fc7860d7d07e admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:03.636273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c74d69e-7974-4c65-a1ec-fc7860d7d07e admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:03.641496 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c7231e-7991-4f94-973e-7a60aa04ba9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:03.642151 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c7231e-7991-4f94-973e-7a60aa04ba9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:03.642434 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c7231e-7991-4f94-973e-7a60aa04ba9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:03.642715 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c7231e-7991-4f94-973e-7a60aa04ba9a admin admin] PATH_INFO: `/v3/projects/97a360e480f84c6c9ab9c946829bb4c0/users/447c9cc3bed142efa7e75dd00b5e4d0e/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:03.657417 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c7231e-7991-4f94-973e-7a60aa04ba9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=97a360e480f84c6c9ab9c946829bb4c0, user_id=447c9cc3bed142efa7e75dd00b5e4d0e, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:03.660159 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c7231e-7991-4f94-973e-7a60aa04ba9a admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:03.711905 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 994/2006] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:32:03 2022] PUT /identity/v3/projects/97a360e480f84c6c9ab9c946829bb4c0/users/447c9cc3bed142efa7e75dd00b5e4d0e/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:32:03.715983 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8956c7-3f6f-4be9-a3ed-21cc2995e4ba admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:03.726104 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8956c7-3f6f-4be9-a3ed-21cc2995e4ba admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:03.770198 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:03.782683 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5763b7c8-8da0-4f26-8af5-84172fe00442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:03.783480 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763b7c8-8da0-4f26-8af5-84172fe00442 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:03.783768 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763b7c8-8da0-4f26-8af5-84172fe00442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:03.784051 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763b7c8-8da0-4f26-8af5-84172fe00442 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:03.784778 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5763b7c8-8da0-4f26-8af5-84172fe00442 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:03.786370 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5763b7c8-8da0-4f26-8af5-84172fe00442 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:03.795900 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1013/2007] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:32:03.800028 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c7231e-7991-4f94-973e-7a60aa04ba9a admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:03.810463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c7231e-7991-4f94-973e-7a60aa04ba9a admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:03.815509 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d5900-2fc3-4ed6-af70-076f2aad75e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:03.816210 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d5900-2fc3-4ed6-af70-076f2aad75e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:03.816606 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d5900-2fc3-4ed6-af70-076f2aad75e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:03.816766 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d5900-2fc3-4ed6-af70-076f2aad75e7 admin admin] PATH_INFO: `/v3/projects/97a360e480f84c6c9ab9c946829bb4c0/users/447c9cc3bed142efa7e75dd00b5e4d0e/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:03.818748 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4d5900-2fc3-4ed6-af70-076f2aad75e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97a360e480f84c6c9ab9c946829bb4c0, user_id=447c9cc3bed142efa7e75dd00b5e4d0e, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:03.821416 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4d5900-2fc3-4ed6-af70-076f2aad75e7 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:03.832043 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 995/2008] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:32:03 2022] PUT /identity/v3/projects/97a360e480f84c6c9ab9c946829bb4c0/users/447c9cc3bed142efa7e75dd00b5e4d0e/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:03.847470 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfeff4-2d44-4331-885d-f4d94aef2c88 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:03.847759 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfeff4-2d44-4331-885d-f4d94aef2c88 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:03.848053 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfeff4-2d44-4331-885d-f4d94aef2c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:03.876661 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-1acfeff4-2d44-4331-885d-f4d94aef2c88 None None] MFA Rules not processed for user `447c9cc3bed142efa7e75dd00b5e4d0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:03.878026 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-1acfeff4-2d44-4331-885d-f4d94aef2c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:03.944815 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1014/2009] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:03 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:03.956868 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d5900-2fc3-4ed6-af70-076f2aad75e7 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:03.958583 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-3e4d5900-2fc3-4ed6-af70-076f2aad75e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:03.972380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:04.024578 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d5900-2fc3-4ed6-af70-076f2aad75e7 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:04.030421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b06a7-a76e-4041-b58f-bbbbddd57770 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:04.036282 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b06a7-a76e-4041-b58f-bbbbddd57770 service cinder] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:04.036282 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b06a7-a76e-4041-b58f-bbbbddd57770 service cinder] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:04.036282 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b06a7-a76e-4041-b58f-bbbbddd57770 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:04.091773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5b06a7-a76e-4041-b58f-bbbbddd57770 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:04.093188 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5b06a7-a76e-4041-b58f-bbbbddd57770 service cinder] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:04.102284 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 996/2010] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:32:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:32:08.369294 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acfeff4-2d44-4331-885d-f4d94aef2c88 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:08.444811 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acfeff4-2d44-4331-885d-f4d94aef2c88 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:08.449483 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b524eee-18fe-45b5-ab66-9c43a845d570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:08.450130 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b524eee-18fe-45b5-ab66-9c43a845d570 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:08.450417 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b524eee-18fe-45b5-ab66-9c43a845d570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:08.450698 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b524eee-18fe-45b5-ab66-9c43a845d570 admin admin] PATH_INFO: `/v3/users/aaa1959b3e0a47359653f9e371361149` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:08.451869 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b524eee-18fe-45b5-ab66-9c43a845d570 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aaa1959b3e0a47359653f9e371361149)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:08.454641 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b524eee-18fe-45b5-ab66-9c43a845d570 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:08.644215 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-8b524eee-18fe-45b5-ab66-9c43a845d570 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aaa1959b3e0a47359653f9e371361149'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:08.644949 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-8b524eee-18fe-45b5-ab66-9c43a845d570 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aaa1959b3e0a47359653f9e371361149'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:08.658230 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-8b524eee-18fe-45b5-ab66-9c43a845d570 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aaa1959b3e0a47359653f9e371361149'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:08.662027 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:08.716847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-8b524eee-18fe-45b5-ab66-9c43a845d570 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aaa1959b3e0a47359653f9e371361149'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:08.725569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-8b524eee-18fe-45b5-ab66-9c43a845d570 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aaa1959b3e0a47359653f9e371361149'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:08.732695 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1015/2011] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:32:08 2022] DELETE /identity/v3/users/aaa1959b3e0a47359653f9e371361149 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:08.737322 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c5cb81-46f4-4895-8b08-de9c990a7a28 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:08.737769 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c5cb81-46f4-4895-8b08-de9c990a7a28 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:08.737914 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c5cb81-46f4-4895-8b08-de9c990a7a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:08.757292 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-d4c5cb81-46f4-4895-8b08-de9c990a7a28 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:08.758545 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-d4c5cb81-46f4-4895-8b08-de9c990a7a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:08.829900 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 997/2012] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:08.839885 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b524eee-18fe-45b5-ab66-9c43a845d570 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:08.841642 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-8b524eee-18fe-45b5-ab66-9c43a845d570 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:08.904536 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b524eee-18fe-45b5-ab66-9c43a845d570 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:08.908343 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674dc9c6-23e6-4c82-a122-1c76216ca876 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:08.908967 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674dc9c6-23e6-4c82-a122-1c76216ca876 service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:08.909253 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674dc9c6-23e6-4c82-a122-1c76216ca876 service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:08.909672 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674dc9c6-23e6-4c82-a122-1c76216ca876 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:08.920195 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674dc9c6-23e6-4c82-a122-1c76216ca876 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:08.922256 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674dc9c6-23e6-4c82-a122-1c76216ca876 service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:08.934237 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1016/2013] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:32:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 15:32:09.443388 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c5cb81-46f4-4895-8b08-de9c990a7a28 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:09.445184 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-d4c5cb81-46f4-4895-8b08-de9c990a7a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:09.461170 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c5cb81-46f4-4895-8b08-de9c990a7a28 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:09.467097 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a9096b-a90d-4b88-b4a4-5ae27472962a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:09.467669 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9096b-a90d-4b88-b4a4-5ae27472962a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:09.467984 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9096b-a90d-4b88-b4a4-5ae27472962a admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:09.468362 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9096b-a90d-4b88-b4a4-5ae27472962a admin admin] PATH_INFO: `/v3/projects/e15350092bd2477dbf952992e58d1bda` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:09.469499 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a9096b-a90d-4b88-b4a4-5ae27472962a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e15350092bd2477dbf952992e58d1bda)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:09.471376 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a9096b-a90d-4b88-b4a4-5ae27472962a admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:09.570314 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-b0a9096b-a90d-4b88-b4a4-5ae27472962a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e15350092bd2477dbf952992e58d1bda', 'request_id': 'req-b0a9096b-a90d-4b88-b4a4-5ae27472962a'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:09.576989 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-b0a9096b-a90d-4b88-b4a4-5ae27472962a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e15350092bd2477dbf952992e58d1bda', 'request_id': 'req-b0a9096b-a90d-4b88-b4a4-5ae27472962a'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:09.651733 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-b0a9096b-a90d-4b88-b4a4-5ae27472962a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e15350092bd2477dbf952992e58d1bda', 'request_id': 'req-b0a9096b-a90d-4b88-b4a4-5ae27472962a'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:09.653521 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:09.668716 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-b0a9096b-a90d-4b88-b4a4-5ae27472962a admin admin] The token cache is being invalidate because project e15350092bd2477dbf952992e58d1bda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:32:09.669043 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-b0a9096b-a90d-4b88-b4a4-5ae27472962a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:09.669665 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-b0a9096b-a90d-4b88-b4a4-5ae27472962a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:09.671830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 998/2014] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:32:09 2022] DELETE /identity/v3/projects/e15350092bd2477dbf952992e58d1bda => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:09.676060 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674dc9c6-23e6-4c82-a122-1c76216ca876 service neutron] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:09.688144 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674dc9c6-23e6-4c82-a122-1c76216ca876 service neutron] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:09.767602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b79629e-9977-4741-ba87-5fe000e424d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:09.768558 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b79629e-9977-4741-ba87-5fe000e424d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:09.768783 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b79629e-9977-4741-ba87-5fe000e424d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:09.769088 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b79629e-9977-4741-ba87-5fe000e424d4 admin admin] PATH_INFO: `/v3/users/447c9cc3bed142efa7e75dd00b5e4d0e` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:09.770637 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b79629e-9977-4741-ba87-5fe000e424d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=447c9cc3bed142efa7e75dd00b5e4d0e)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:09.772865 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b79629e-9977-4741-ba87-5fe000e424d4 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:09.959971 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-2b79629e-9977-4741-ba87-5fe000e424d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '447c9cc3bed142efa7e75dd00b5e4d0e'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:09.960799 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-2b79629e-9977-4741-ba87-5fe000e424d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '447c9cc3bed142efa7e75dd00b5e4d0e'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:09.972761 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-2b79629e-9977-4741-ba87-5fe000e424d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '447c9cc3bed142efa7e75dd00b5e4d0e'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:10.075439 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-2b79629e-9977-4741-ba87-5fe000e424d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '447c9cc3bed142efa7e75dd00b5e4d0e'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:10.084179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-2b79629e-9977-4741-ba87-5fe000e424d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '447c9cc3bed142efa7e75dd00b5e4d0e'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:10.095469 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1017/2015] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:32:09 2022] DELETE /identity/v3/users/447c9cc3bed142efa7e75dd00b5e4d0e => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:10.737055 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a9096b-a90d-4b88-b4a4-5ae27472962a admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:10.738658 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-b0a9096b-a90d-4b88-b4a4-5ae27472962a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:10.762128 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a9096b-a90d-4b88-b4a4-5ae27472962a admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:10.843146 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa87ec3-e219-4394-8c86-fc50b6849f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:10.843817 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa87ec3-e219-4394-8c86-fc50b6849f3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:10.844117 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa87ec3-e219-4394-8c86-fc50b6849f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:10.844409 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa87ec3-e219-4394-8c86-fc50b6849f3e admin admin] PATH_INFO: `/v3/projects/97a360e480f84c6c9ab9c946829bb4c0` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:10.858521 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa87ec3-e219-4394-8c86-fc50b6849f3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=97a360e480f84c6c9ab9c946829bb4c0)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:10.859308 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa87ec3-e219-4394-8c86-fc50b6849f3e admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:10.970109 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-1aa87ec3-e219-4394-8c86-fc50b6849f3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97a360e480f84c6c9ab9c946829bb4c0', 'request_id': 'req-1aa87ec3-e219-4394-8c86-fc50b6849f3e'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:10.977107 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-1aa87ec3-e219-4394-8c86-fc50b6849f3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97a360e480f84c6c9ab9c946829bb4c0', 'request_id': 'req-1aa87ec3-e219-4394-8c86-fc50b6849f3e'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:11.094781 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-1aa87ec3-e219-4394-8c86-fc50b6849f3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97a360e480f84c6c9ab9c946829bb4c0', 'request_id': 'req-1aa87ec3-e219-4394-8c86-fc50b6849f3e'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:11.110557 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-1aa87ec3-e219-4394-8c86-fc50b6849f3e admin admin] The token cache is being invalidate because project 97a360e480f84c6c9ab9c946829bb4c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:32:11.111075 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-1aa87ec3-e219-4394-8c86-fc50b6849f3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:11.112517 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-1aa87ec3-e219-4394-8c86-fc50b6849f3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:11.113725 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 999/2016] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:32:10 2022] DELETE /identity/v3/projects/97a360e480f84c6c9ab9c946829bb4c0 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:32:11.119966 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6822a4-f890-4455-aacd-a7387d3582ed None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:11.120568 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6822a4-f890-4455-aacd-a7387d3582ed None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:11.121068 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6822a4-f890-4455-aacd-a7387d3582ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:11.145153 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-3e6822a4-f890-4455-aacd-a7387d3582ed None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:11.146662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-3e6822a4-f890-4455-aacd-a7387d3582ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:11.151779 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:11.163332 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa87ec3-e219-4394-8c86-fc50b6849f3e admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:11.164956 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-1aa87ec3-e219-4394-8c86-fc50b6849f3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:11.232247 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1018/2017] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:11.238338 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6c7ff-0c0a-4d79-b23b-b81de98ad060 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:11.238857 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6c7ff-0c0a-4d79-b23b-b81de98ad060 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:11.239147 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6c7ff-0c0a-4d79-b23b-b81de98ad060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:11.259789 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-55c6c7ff-0c0a-4d79-b23b-b81de98ad060 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:11.261678 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-55c6c7ff-0c0a-4d79-b23b-b81de98ad060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:11.265022 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa87ec3-e219-4394-8c86-fc50b6849f3e admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:11.269897 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2711b86a-f3b4-41b0-94fa-5ed293198272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:11.270619 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711b86a-f3b4-41b0-94fa-5ed293198272 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:11.270715 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1019/2018] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:11.271059 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711b86a-f3b4-41b0-94fa-5ed293198272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:11.271461 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711b86a-f3b4-41b0-94fa-5ed293198272 admin admin] PATH_INFO: `/v3/users/b0882cb9bf4744feae298fb8f10cebeb` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:11.272807 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2711b86a-f3b4-41b0-94fa-5ed293198272 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0882cb9bf4744feae298fb8f10cebeb)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:11.274825 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c6c7ff-0c0a-4d79-b23b-b81de98ad060 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:11.274938 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2711b86a-f3b4-41b0-94fa-5ed293198272 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:11.284424 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c6c7ff-0c0a-4d79-b23b-b81de98ad060 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:11.289210 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42235f3c-4e2c-408a-9ce6-92f9387f7b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:11.289824 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42235f3c-4e2c-408a-9ce6-92f9387f7b91 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:11.290110 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42235f3c-4e2c-408a-9ce6-92f9387f7b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:11.290387 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42235f3c-4e2c-408a-9ce6-92f9387f7b91 admin admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:11.291081 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42235f3c-4e2c-408a-9ce6-92f9387f7b91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:32:11.291389 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42235f3c-4e2c-408a-9ce6-92f9387f7b91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:11.293022 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42235f3c-4e2c-408a-9ce6-92f9387f7b91 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:11.298986 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:11.308982 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1020/2019] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:32:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:32:11.313286 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42235f3c-4e2c-408a-9ce6-92f9387f7b91 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:11.323465 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42235f3c-4e2c-408a-9ce6-92f9387f7b91 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:11.329374 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b510ac-8462-41a3-91dd-d8939fef503e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:11.330121 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b510ac-8462-41a3-91dd-d8939fef503e admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:11.330470 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b510ac-8462-41a3-91dd-d8939fef503e admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:11.330838 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b510ac-8462-41a3-91dd-d8939fef503e admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:11.331577 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b510ac-8462-41a3-91dd-d8939fef503e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:11.333543 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b510ac-8462-41a3-91dd-d8939fef503e admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:11.426133 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1021/2020] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:32:11 2022] POST /identity/v3/projects => generated 363 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:32:11.430146 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b510ac-8462-41a3-91dd-d8939fef503e admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:11.432271 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-2711b86a-f3b4-41b0-94fa-5ed293198272 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0882cb9bf4744feae298fb8f10cebeb'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:11.439828 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-2711b86a-f3b4-41b0-94fa-5ed293198272 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0882cb9bf4744feae298fb8f10cebeb'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:11.440582 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-2711b86a-f3b4-41b0-94fa-5ed293198272 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0882cb9bf4744feae298fb8f10cebeb'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:11.454422 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-2711b86a-f3b4-41b0-94fa-5ed293198272 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0882cb9bf4744feae298fb8f10cebeb'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:11.455128 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-2711b86a-f3b4-41b0-94fa-5ed293198272 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0882cb9bf4744feae298fb8f10cebeb'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:11.493362 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b510ac-8462-41a3-91dd-d8939fef503e admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:11.498214 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985fac3d-1b38-4467-8b3e-ec5dd43b26ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:11.500217 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985fac3d-1b38-4467-8b3e-ec5dd43b26ce admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:11.501267 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985fac3d-1b38-4467-8b3e-ec5dd43b26ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:11.501549 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985fac3d-1b38-4467-8b3e-ec5dd43b26ce admin admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:11.502209 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985fac3d-1b38-4467-8b3e-ec5dd43b26ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:11.504271 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985fac3d-1b38-4467-8b3e-ec5dd43b26ce admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:11.533918 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1000/2021] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:32:11 2022] DELETE /identity/v3/users/b0882cb9bf4744feae298fb8f10cebeb => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:32:11.616184 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1022/2022] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:32:11 2022] POST /identity/v3/users => generated 282 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:32:11.619652 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2711b86a-f3b4-41b0-94fa-5ed293198272 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:11.621246 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-2711b86a-f3b4-41b0-94fa-5ed293198272 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:11.693024 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2711b86a-f3b4-41b0-94fa-5ed293198272 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:11.697648 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7ea11d-f9b4-44cc-b11d-d9a93a7a9e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:11.698269 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ea11d-f9b4-44cc-b11d-d9a93a7a9e65 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:11.698579 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ea11d-f9b4-44cc-b11d-d9a93a7a9e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:11.698855 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ea11d-f9b4-44cc-b11d-d9a93a7a9e65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:11.699550 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7ea11d-f9b4-44cc-b11d-d9a93a7a9e65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:11.701193 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7ea11d-f9b4-44cc-b11d-d9a93a7a9e65 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:11.712555 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1001/2023] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:32:11.716323 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985fac3d-1b38-4467-8b3e-ec5dd43b26ce admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:11.731523 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985fac3d-1b38-4467-8b3e-ec5dd43b26ce admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:11.731523 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab9660-7331-4a83-91f4-eaa1d43fde88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:11.732129 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab9660-7331-4a83-91f4-eaa1d43fde88 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:11.732415 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab9660-7331-4a83-91f4-eaa1d43fde88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:11.732697 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab9660-7331-4a83-91f4-eaa1d43fde88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:11.733954 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ab9660-7331-4a83-91f4-eaa1d43fde88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:11.735545 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ab9660-7331-4a83-91f4-eaa1d43fde88 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:11.746161 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1023/2024] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:32:11.751041 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7ea11d-f9b4-44cc-b11d-d9a93a7a9e65 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:11.761651 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7ea11d-f9b4-44cc-b11d-d9a93a7a9e65 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:11.766668 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc958a-d3f4-4e25-b68b-d7ae50404993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:11.767385 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc958a-d3f4-4e25-b68b-d7ae50404993 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:11.767667 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc958a-d3f4-4e25-b68b-d7ae50404993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:11.767949 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc958a-d3f4-4e25-b68b-d7ae50404993 admin admin] PATH_INFO: `/v3/projects/80965fbb5f754555b141b537a6f266d9/users/f2838c08a74340b98e61b6834e8753bc/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:11.783635 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc958a-d3f4-4e25-b68b-d7ae50404993 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80965fbb5f754555b141b537a6f266d9, user_id=f2838c08a74340b98e61b6834e8753bc, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:11.786321 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc958a-d3f4-4e25-b68b-d7ae50404993 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:11.792004 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab9660-7331-4a83-91f4-eaa1d43fde88 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:11.794082 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-b2ab9660-7331-4a83-91f4-eaa1d43fde88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:11.814055 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab9660-7331-4a83-91f4-eaa1d43fde88 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:11.819527 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316269f8-20c0-4cff-9238-f2cc2584b847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:11.821033 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316269f8-20c0-4cff-9238-f2cc2584b847 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:11.821335 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316269f8-20c0-4cff-9238-f2cc2584b847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:11.822199 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316269f8-20c0-4cff-9238-f2cc2584b847 admin admin] PATH_INFO: `/v3/projects/020b2f0e82d14211b286f1ac1e3823a9` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:11.823848 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316269f8-20c0-4cff-9238-f2cc2584b847 admin admin] RBAC: Authorizing `identity:delete_project(project_id=020b2f0e82d14211b286f1ac1e3823a9)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:11.826187 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316269f8-20c0-4cff-9238-f2cc2584b847 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:11.868402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1002/2025] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:32:11 2022] PUT /identity/v3/projects/80965fbb5f754555b141b537a6f266d9/users/f2838c08a74340b98e61b6834e8753bc/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:11.878934 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a01857-b0b3-4b78-af36-9c046151e76a None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:11.879094 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a01857-b0b3-4b78-af36-9c046151e76a None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:11.879558 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a01857-b0b3-4b78-af36-9c046151e76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:11.911968 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-39a01857-b0b3-4b78-af36-9c046151e76a None None] MFA Rules not processed for user `f2838c08a74340b98e61b6834e8753bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:11.913225 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-39a01857-b0b3-4b78-af36-9c046151e76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:11.966063 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:11.975545 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-316269f8-20c0-4cff-9238-f2cc2584b847 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '020b2f0e82d14211b286f1ac1e3823a9', 'request_id': 'req-316269f8-20c0-4cff-9238-f2cc2584b847'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:11.984442 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-316269f8-20c0-4cff-9238-f2cc2584b847 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '020b2f0e82d14211b286f1ac1e3823a9', 'request_id': 'req-316269f8-20c0-4cff-9238-f2cc2584b847'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:11.988542 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:11.990980 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-316269f8-20c0-4cff-9238-f2cc2584b847 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '020b2f0e82d14211b286f1ac1e3823a9', 'request_id': 'req-316269f8-20c0-4cff-9238-f2cc2584b847'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:11.992713 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1003/2026] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:32:12.003745 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a01857-b0b3-4b78-af36-9c046151e76a None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:12.005113 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-39a01857-b0b3-4b78-af36-9c046151e76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:12.065134 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a01857-b0b3-4b78-af36-9c046151e76a None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:12.081966 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-316269f8-20c0-4cff-9238-f2cc2584b847 admin admin] The token cache is being invalidate because project 020b2f0e82d14211b286f1ac1e3823a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:32:12.082485 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-316269f8-20c0-4cff-9238-f2cc2584b847 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:12.083203 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-316269f8-20c0-4cff-9238-f2cc2584b847 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:12.084910 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1024/2027] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:32:11 2022] DELETE /identity/v3/projects/020b2f0e82d14211b286f1ac1e3823a9 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:12.089699 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ad67f-b641-48af-b80e-884861f7ebeb None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:12.090075 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ad67f-b641-48af-b80e-884861f7ebeb None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:12.090355 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ad67f-b641-48af-b80e-884861f7ebeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:12.110955 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-281ad67f-b641-48af-b80e-884861f7ebeb None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:12.112226 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-281ad67f-b641-48af-b80e-884861f7ebeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:12.115889 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e20f8-c587-4b0e-b0da-80a0dc6a1045 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:12.116540 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e20f8-c587-4b0e-b0da-80a0dc6a1045 service cinder] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:12.116826 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e20f8-c587-4b0e-b0da-80a0dc6a1045 service cinder] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:12.117109 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e20f8-c587-4b0e-b0da-80a0dc6a1045 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:12.119374 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-6f4e20f8-c587-4b0e-b0da-80a0dc6a1045 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:12.175420 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1025/2028] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:12.184408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19075c3d-0428-44f8-b621-1b3bde9e77af None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:12.184916 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19075c3d-0428-44f8-b621-1b3bde9e77af None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:12.185206 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19075c3d-0428-44f8-b621-1b3bde9e77af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:12.193414 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4e20f8-c587-4b0e-b0da-80a0dc6a1045 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:12.194888 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4e20f8-c587-4b0e-b0da-80a0dc6a1045 service cinder] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:12.204950 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1004/2029] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:32:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 15:32:12.205619 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-19075c3d-0428-44f8-b621-1b3bde9e77af None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:12.206805 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-19075c3d-0428-44f8-b621-1b3bde9e77af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:12.215034 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1026/2030] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:32:12.219953 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e20f8-c587-4b0e-b0da-80a0dc6a1045 service cinder] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:12.230615 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e20f8-c587-4b0e-b0da-80a0dc6a1045 service cinder] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:12.235599 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc8ba3-f2e2-40f4-8281-47e46264d682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:12.236237 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc8ba3-f2e2-40f4-8281-47e46264d682 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:12.236573 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc8ba3-f2e2-40f4-8281-47e46264d682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:12.236854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc8ba3-f2e2-40f4-8281-47e46264d682 admin admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:12.237540 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc8ba3-f2e2-40f4-8281-47e46264d682 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:32:12.237844 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc8ba3-f2e2-40f4-8281-47e46264d682 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:12.239419 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc8ba3-f2e2-40f4-8281-47e46264d682 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:12.251578 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1005/2031] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:32:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:32:12.256038 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19075c3d-0428-44f8-b621-1b3bde9e77af None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:12.266146 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19075c3d-0428-44f8-b621-1b3bde9e77af None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:12.270961 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af77fbb3-9e9d-4f36-9f2e-3d1566365860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:12.271575 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af77fbb3-9e9d-4f36-9f2e-3d1566365860 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:12.271859 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af77fbb3-9e9d-4f36-9f2e-3d1566365860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:12.272169 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af77fbb3-9e9d-4f36-9f2e-3d1566365860 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:12.272834 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af77fbb3-9e9d-4f36-9f2e-3d1566365860 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:12.274738 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af77fbb3-9e9d-4f36-9f2e-3d1566365860 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:12.363973 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1027/2032] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:32:12 2022] POST /identity/v3/projects => generated 365 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:32:12.368362 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc8ba3-f2e2-40f4-8281-47e46264d682 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:12.378392 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc8ba3-f2e2-40f4-8281-47e46264d682 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:12.435860 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7690d4bd-2b63-4347-91bd-c17f3ea3f1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:12.436535 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690d4bd-2b63-4347-91bd-c17f3ea3f1a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:12.436881 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690d4bd-2b63-4347-91bd-c17f3ea3f1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:12.440560 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690d4bd-2b63-4347-91bd-c17f3ea3f1a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:12.440560 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7690d4bd-2b63-4347-91bd-c17f3ea3f1a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:12.440560 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7690d4bd-2b63-4347-91bd-c17f3ea3f1a1 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:12.545476 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1006/2033] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:32:12 2022] POST /identity/v3/users => generated 283 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:32:12.549770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af77fbb3-9e9d-4f36-9f2e-3d1566365860 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:12.560246 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af77fbb3-9e9d-4f36-9f2e-3d1566365860 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:12.565061 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65676326-e1ee-4286-9b9f-38839acb86f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:12.565705 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65676326-e1ee-4286-9b9f-38839acb86f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:12.565988 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65676326-e1ee-4286-9b9f-38839acb86f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:12.566267 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65676326-e1ee-4286-9b9f-38839acb86f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:12.567042 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65676326-e1ee-4286-9b9f-38839acb86f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:12.568772 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65676326-e1ee-4286-9b9f-38839acb86f8 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:12.579794 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1028/2034] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:32:12.583746 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7690d4bd-2b63-4347-91bd-c17f3ea3f1a1 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:12.594772 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7690d4bd-2b63-4347-91bd-c17f3ea3f1a1 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:12.599602 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b3b0c4-2b45-4b47-b3c6-483e3521e072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:12.600252 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3b0c4-2b45-4b47-b3c6-483e3521e072 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:12.600594 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3b0c4-2b45-4b47-b3c6-483e3521e072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:12.600900 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3b0c4-2b45-4b47-b3c6-483e3521e072 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:12.601915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b3b0c4-2b45-4b47-b3c6-483e3521e072 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:12.603285 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b3b0c4-2b45-4b47-b3c6-483e3521e072 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:12.614241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1007/2035] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:32:12.618779 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65676326-e1ee-4286-9b9f-38839acb86f8 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:12.629462 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65676326-e1ee-4286-9b9f-38839acb86f8 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:12.634254 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d423cdfe-fca9-4a7e-ab17-80cfb8aa5407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:12.634955 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d423cdfe-fca9-4a7e-ab17-80cfb8aa5407 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:12.635246 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d423cdfe-fca9-4a7e-ab17-80cfb8aa5407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:12.635525 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d423cdfe-fca9-4a7e-ab17-80cfb8aa5407 admin admin] PATH_INFO: `/v3/projects/ebcbc3b27bf04b49a224a246a7ccb5c8/users/3ee3e74f242748059404f8af83362497/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:12.650556 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d423cdfe-fca9-4a7e-ab17-80cfb8aa5407 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebcbc3b27bf04b49a224a246a7ccb5c8, user_id=3ee3e74f242748059404f8af83362497, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:12.653367 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d423cdfe-fca9-4a7e-ab17-80cfb8aa5407 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:12.653975 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:12.745460 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1029/2036] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:32:12 2022] PUT /identity/v3/projects/ebcbc3b27bf04b49a224a246a7ccb5c8/users/3ee3e74f242748059404f8af83362497/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:12.756010 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1559173b-638d-465f-98a6-2c9a993249cf None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:12.756512 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1559173b-638d-465f-98a6-2c9a993249cf None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:12.756604 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1559173b-638d-465f-98a6-2c9a993249cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:12.791554 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-1559173b-638d-465f-98a6-2c9a993249cf None None] MFA Rules not processed for user `3ee3e74f242748059404f8af83362497`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:12.792873 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-1559173b-638d-465f-98a6-2c9a993249cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:12.859004 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1008/2037] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:12.866297 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d423cdfe-fca9-4a7e-ab17-80cfb8aa5407 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:12.938145 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d423cdfe-fca9-4a7e-ab17-80cfb8aa5407 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:12.944009 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0711c305-261a-4494-8eea-4a0e403ea012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:12.944680 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0711c305-261a-4494-8eea-4a0e403ea012 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:12.944941 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0711c305-261a-4494-8eea-4a0e403ea012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:12.945227 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0711c305-261a-4494-8eea-4a0e403ea012 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:12.945895 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0711c305-261a-4494-8eea-4a0e403ea012 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:12.947896 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0711c305-261a-4494-8eea-4a0e403ea012 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:12.961792 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:13.086014 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1030/2038] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:32:12 2022] POST /identity/v3/projects => generated 365 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:32:13.090160 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1559173b-638d-465f-98a6-2c9a993249cf None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:13.100986 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1559173b-638d-465f-98a6-2c9a993249cf None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:13.160597 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48f83f1-18d0-40f0-b4cd-135f92768e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:13.161220 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f83f1-18d0-40f0-b4cd-135f92768e54 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:13.161506 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f83f1-18d0-40f0-b4cd-135f92768e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:13.161782 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f83f1-18d0-40f0-b4cd-135f92768e54 admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:13.162435 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48f83f1-18d0-40f0-b4cd-135f92768e54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:13.164366 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48f83f1-18d0-40f0-b4cd-135f92768e54 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:13.259645 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1009/2039] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:32:13 2022] POST /identity/v3/users => generated 289 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:32:13.263607 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0711c305-261a-4494-8eea-4a0e403ea012 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:13.274122 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0711c305-261a-4494-8eea-4a0e403ea012 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:13.279109 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daecb4f8-2d10-4c17-9228-72ed5b5e9073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:13.279753 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daecb4f8-2d10-4c17-9228-72ed5b5e9073 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:13.280038 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daecb4f8-2d10-4c17-9228-72ed5b5e9073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:13.280355 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daecb4f8-2d10-4c17-9228-72ed5b5e9073 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:13.281054 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daecb4f8-2d10-4c17-9228-72ed5b5e9073 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:13.282638 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daecb4f8-2d10-4c17-9228-72ed5b5e9073 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:13.292616 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1031/2040] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:32:13.297102 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48f83f1-18d0-40f0-b4cd-135f92768e54 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:13.312411 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48f83f1-18d0-40f0-b4cd-135f92768e54 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:13.314122 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eef69d-0495-40a8-a939-1644f72fc4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:13.314797 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eef69d-0495-40a8-a939-1644f72fc4a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:13.315085 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eef69d-0495-40a8-a939-1644f72fc4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:13.315367 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eef69d-0495-40a8-a939-1644f72fc4a2 admin admin] PATH_INFO: `/v3/projects/7e75583df48248818e8c19308152fdbc/users/3b3e2822d0164644b12a0c606b7b848c/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:13.331947 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eef69d-0495-40a8-a939-1644f72fc4a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e75583df48248818e8c19308152fdbc, user_id=3b3e2822d0164644b12a0c606b7b848c, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:13.334826 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eef69d-0495-40a8-a939-1644f72fc4a2 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:13.391260 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1010/2041] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:32:13 2022] PUT /identity/v3/projects/7e75583df48248818e8c19308152fdbc/users/3b3e2822d0164644b12a0c606b7b848c/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:13.395341 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daecb4f8-2d10-4c17-9228-72ed5b5e9073 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:13.405511 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daecb4f8-2d10-4c17-9228-72ed5b5e9073 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:13.469555 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb76b27-8a44-4e4b-90f2-3f07ceb98dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:13.473039 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb76b27-8a44-4e4b-90f2-3f07ceb98dda admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:13.473039 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb76b27-8a44-4e4b-90f2-3f07ceb98dda admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:13.473039 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb76b27-8a44-4e4b-90f2-3f07ceb98dda admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:13.473039 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb76b27-8a44-4e4b-90f2-3f07ceb98dda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:13.473360 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb76b27-8a44-4e4b-90f2-3f07ceb98dda admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:13.483169 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1032/2042] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:32:13.487386 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eef69d-0495-40a8-a939-1644f72fc4a2 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:13.498132 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eef69d-0495-40a8-a939-1644f72fc4a2 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:13.503252 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424510aa-2051-48b1-bd78-48c2454d0668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:13.503955 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424510aa-2051-48b1-bd78-48c2454d0668 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:13.504272 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424510aa-2051-48b1-bd78-48c2454d0668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:13.504562 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424510aa-2051-48b1-bd78-48c2454d0668 admin admin] PATH_INFO: `/v3/projects/7e75583df48248818e8c19308152fdbc/users/3b3e2822d0164644b12a0c606b7b848c/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:13.506505 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424510aa-2051-48b1-bd78-48c2454d0668 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e75583df48248818e8c19308152fdbc, user_id=3b3e2822d0164644b12a0c606b7b848c, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:13.509215 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424510aa-2051-48b1-bd78-48c2454d0668 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:13.521623 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1011/2043] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:32:13 2022] PUT /identity/v3/projects/7e75583df48248818e8c19308152fdbc/users/3b3e2822d0164644b12a0c606b7b848c/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:32:13.534116 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73b2cc-cff4-4f64-aac9-ca99399a47d3 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:13.534439 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73b2cc-cff4-4f64-aac9-ca99399a47d3 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:13.534906 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73b2cc-cff4-4f64-aac9-ca99399a47d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:13.565130 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-9c73b2cc-cff4-4f64-aac9-ca99399a47d3 None None] MFA Rules not processed for user `3b3e2822d0164644b12a0c606b7b848c`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:13.565450 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-9c73b2cc-cff4-4f64-aac9-ca99399a47d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:13.652339 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1033/2044] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:13 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:13.661630 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424510aa-2051-48b1-bd78-48c2454d0668 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:13.663352 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-424510aa-2051-48b1-bd78-48c2454d0668 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:13.685164 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:13.721942 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424510aa-2051-48b1-bd78-48c2454d0668 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:13.725665 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea6d10-a30b-4686-bb15-9051d7fc7281 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:13.726282 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea6d10-a30b-4686-bb15-9051d7fc7281 service cinder] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:13.726569 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea6d10-a30b-4686-bb15-9051d7fc7281 service cinder] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:13.726849 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea6d10-a30b-4686-bb15-9051d7fc7281 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:13.736297 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aea6d10-a30b-4686-bb15-9051d7fc7281 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:13.737724 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aea6d10-a30b-4686-bb15-9051d7fc7281 service cinder] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:13.749458 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1012/2045] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:32:13 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:32:14.057683 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73b2cc-cff4-4f64-aac9-ca99399a47d3 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:14.068333 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73b2cc-cff4-4f64-aac9-ca99399a47d3 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:14.072173 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fee455c-20e7-4f8f-b95d-a381eb79e4d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:14.072789 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee455c-20e7-4f8f-b95d-a381eb79e4d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:14.073072 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee455c-20e7-4f8f-b95d-a381eb79e4d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:14.073348 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee455c-20e7-4f8f-b95d-a381eb79e4d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:14.138388 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fee455c-20e7-4f8f-b95d-a381eb79e4d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:14.140403 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fee455c-20e7-4f8f-b95d-a381eb79e4d1 service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:14.145616 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:14.150653 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1034/2046] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:32:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 15:32:15.736762 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea6d10-a30b-4686-bb15-9051d7fc7281 service cinder] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:15.738349 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-7aea6d10-a30b-4686-bb15-9051d7fc7281 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:15.812105 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea6d10-a30b-4686-bb15-9051d7fc7281 service cinder] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:15.818488 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3afb4-8d38-45fc-9c57-2b1a8faaf148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:15.825268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3afb4-8d38-45fc-9c57-2b1a8faaf148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:15.825268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3afb4-8d38-45fc-9c57-2b1a8faaf148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:15.825268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3afb4-8d38-45fc-9c57-2b1a8faaf148 admin admin] PATH_INFO: `/v3/users/01d02b2aecaf4872a947e07c684e6642` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:15.825268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd3afb4-8d38-45fc-9c57-2b1a8faaf148 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01d02b2aecaf4872a947e07c684e6642)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:15.825268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd3afb4-8d38-45fc-9c57-2b1a8faaf148 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:16.039028 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-0fd3afb4-8d38-45fc-9c57-2b1a8faaf148 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01d02b2aecaf4872a947e07c684e6642'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:16.046417 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-0fd3afb4-8d38-45fc-9c57-2b1a8faaf148 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01d02b2aecaf4872a947e07c684e6642'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:16.047111 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-0fd3afb4-8d38-45fc-9c57-2b1a8faaf148 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01d02b2aecaf4872a947e07c684e6642'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:16.064486 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-0fd3afb4-8d38-45fc-9c57-2b1a8faaf148 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01d02b2aecaf4872a947e07c684e6642'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:16.065170 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-0fd3afb4-8d38-45fc-9c57-2b1a8faaf148 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01d02b2aecaf4872a947e07c684e6642'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:16.151298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1013/2047] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:32:15 2022] DELETE /identity/v3/users/01d02b2aecaf4872a947e07c684e6642 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:16.155929 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850a49b-040a-4d27-9e9d-29ae870dffed None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:16.156422 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850a49b-040a-4d27-9e9d-29ae870dffed None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:16.156583 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850a49b-040a-4d27-9e9d-29ae870dffed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:16.176948 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-4850a49b-040a-4d27-9e9d-29ae870dffed None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:16.178176 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-4850a49b-040a-4d27-9e9d-29ae870dffed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:16.244119 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1035/2048] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:16.254432 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3afb4-8d38-45fc-9c57-2b1a8faaf148 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:16.255848 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-0fd3afb4-8d38-45fc-9c57-2b1a8faaf148 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:16.299106 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:16.313202 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3afb4-8d38-45fc-9c57-2b1a8faaf148 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:16.323206 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25918d7b-a849-4319-be7a-6cdcbd78f563 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:16.323206 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25918d7b-a849-4319-be7a-6cdcbd78f563 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:16.323206 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25918d7b-a849-4319-be7a-6cdcbd78f563 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:16.323206 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25918d7b-a849-4319-be7a-6cdcbd78f563 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:16.327791 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25918d7b-a849-4319-be7a-6cdcbd78f563 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:16.329312 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25918d7b-a849-4319-be7a-6cdcbd78f563 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:16.339704 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1014/2049] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:32:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:32:16.858794 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850a49b-040a-4d27-9e9d-29ae870dffed None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:16.860353 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-4850a49b-040a-4d27-9e9d-29ae870dffed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:16.873789 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850a49b-040a-4d27-9e9d-29ae870dffed None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:16.881252 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d6b2b-a430-4e29-81a0-d39a475b2635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:16.882014 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d6b2b-a430-4e29-81a0-d39a475b2635 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:16.883274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d6b2b-a430-4e29-81a0-d39a475b2635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:16.883274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d6b2b-a430-4e29-81a0-d39a475b2635 admin admin] PATH_INFO: `/v3/projects/d24c6c847bc04921b7da2949dc1222d2` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:16.883849 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201d6b2b-a430-4e29-81a0-d39a475b2635 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d24c6c847bc04921b7da2949dc1222d2)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:16.885930 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201d6b2b-a430-4e29-81a0-d39a475b2635 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:16.997264 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-201d6b2b-a430-4e29-81a0-d39a475b2635 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd24c6c847bc04921b7da2949dc1222d2', 'request_id': 'req-201d6b2b-a430-4e29-81a0-d39a475b2635'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:17.003575 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-201d6b2b-a430-4e29-81a0-d39a475b2635 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd24c6c847bc04921b7da2949dc1222d2', 'request_id': 'req-201d6b2b-a430-4e29-81a0-d39a475b2635'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:17.009372 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-201d6b2b-a430-4e29-81a0-d39a475b2635 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd24c6c847bc04921b7da2949dc1222d2', 'request_id': 'req-201d6b2b-a430-4e29-81a0-d39a475b2635'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:17.100531 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-201d6b2b-a430-4e29-81a0-d39a475b2635 admin admin] The token cache is being invalidate because project d24c6c847bc04921b7da2949dc1222d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:32:17.100531 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-201d6b2b-a430-4e29-81a0-d39a475b2635 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:17.101135 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-201d6b2b-a430-4e29-81a0-d39a475b2635 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:17.103206 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1036/2050] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:32:16 2022] DELETE /identity/v3/projects/d24c6c847bc04921b7da2949dc1222d2 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:17.106849 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25918d7b-a849-4319-be7a-6cdcbd78f563 service neutron] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:17.116846 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25918d7b-a849-4319-be7a-6cdcbd78f563 service neutron] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:17.183523 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae835a3-36ac-4d48-b9cb-9797b250c92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:17.184194 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae835a3-36ac-4d48-b9cb-9797b250c92d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:17.184477 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae835a3-36ac-4d48-b9cb-9797b250c92d admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:17.184753 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae835a3-36ac-4d48-b9cb-9797b250c92d admin admin] PATH_INFO: `/v3/users/9203e33c023c401fb4d6cc2d686c0175` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:17.186446 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae835a3-36ac-4d48-b9cb-9797b250c92d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9203e33c023c401fb4d6cc2d686c0175)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:17.188442 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae835a3-36ac-4d48-b9cb-9797b250c92d admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:17.370360 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-cae835a3-36ac-4d48-b9cb-9797b250c92d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9203e33c023c401fb4d6cc2d686c0175'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:17.377185 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-cae835a3-36ac-4d48-b9cb-9797b250c92d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9203e33c023c401fb4d6cc2d686c0175'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:17.377873 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-cae835a3-36ac-4d48-b9cb-9797b250c92d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9203e33c023c401fb4d6cc2d686c0175'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:17.390283 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-cae835a3-36ac-4d48-b9cb-9797b250c92d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9203e33c023c401fb4d6cc2d686c0175'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:17.390928 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-cae835a3-36ac-4d48-b9cb-9797b250c92d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9203e33c023c401fb4d6cc2d686c0175'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:17.464471 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1015/2051] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:32:17 2022] DELETE /identity/v3/users/9203e33c023c401fb4d6cc2d686c0175 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:17.799148 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d6b2b-a430-4e29-81a0-d39a475b2635 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:17.800835 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-201d6b2b-a430-4e29-81a0-d39a475b2635 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:17.814853 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d6b2b-a430-4e29-81a0-d39a475b2635 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:17.870279 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae835a3-36ac-4d48-b9cb-9797b250c92d admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:17.873264 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-cae835a3-36ac-4d48-b9cb-9797b250c92d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:17.880967 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f382f02-4448-4766-bc25-a5ffd13e3d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:17.881633 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f382f02-4448-4766-bc25-a5ffd13e3d8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:17.881915 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f382f02-4448-4766-bc25-a5ffd13e3d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:17.882194 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f382f02-4448-4766-bc25-a5ffd13e3d8c admin admin] PATH_INFO: `/v3/users/3ee3e74f242748059404f8af83362497` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:17.884855 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f382f02-4448-4766-bc25-a5ffd13e3d8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ee3e74f242748059404f8af83362497)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:17.889286 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f382f02-4448-4766-bc25-a5ffd13e3d8c admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:17.898690 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae835a3-36ac-4d48-b9cb-9797b250c92d admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:17.908667 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7918c08-6d54-4e1a-a956-ea0f2648e233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:17.909350 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7918c08-6d54-4e1a-a956-ea0f2648e233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:17.909631 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7918c08-6d54-4e1a-a956-ea0f2648e233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:17.909959 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7918c08-6d54-4e1a-a956-ea0f2648e233 admin admin] PATH_INFO: `/v3/projects/1140834102cf47a88a14145e62f8e14c` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:17.911255 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7918c08-6d54-4e1a-a956-ea0f2648e233 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1140834102cf47a88a14145e62f8e14c)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:17.913282 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7918c08-6d54-4e1a-a956-ea0f2648e233 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:18.014823 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:18.119377 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-8f382f02-4448-4766-bc25-a5ffd13e3d8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ee3e74f242748059404f8af83362497'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:18.120080 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-8f382f02-4448-4766-bc25-a5ffd13e3d8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ee3e74f242748059404f8af83362497'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:18.120287 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-f7918c08-6d54-4e1a-a956-ea0f2648e233 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1140834102cf47a88a14145e62f8e14c', 'request_id': 'req-f7918c08-6d54-4e1a-a956-ea0f2648e233'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:18.132518 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-f7918c08-6d54-4e1a-a956-ea0f2648e233 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1140834102cf47a88a14145e62f8e14c', 'request_id': 'req-f7918c08-6d54-4e1a-a956-ea0f2648e233'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:18.133321 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-8f382f02-4448-4766-bc25-a5ffd13e3d8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ee3e74f242748059404f8af83362497'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:18.136957 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:18.230767 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-f7918c08-6d54-4e1a-a956-ea0f2648e233 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1140834102cf47a88a14145e62f8e14c', 'request_id': 'req-f7918c08-6d54-4e1a-a956-ea0f2648e233'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:18.241945 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-f7918c08-6d54-4e1a-a956-ea0f2648e233 admin admin] The token cache is being invalidate because project 1140834102cf47a88a14145e62f8e14c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:32:18.242234 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-f7918c08-6d54-4e1a-a956-ea0f2648e233 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:18.242849 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-f7918c08-6d54-4e1a-a956-ea0f2648e233 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:18.244399 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1016/2052] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:32:17 2022] DELETE /identity/v3/projects/1140834102cf47a88a14145e62f8e14c => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:32:18.248955 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc0fc1-ffc4-45fb-b0f0-996c7ba645fc None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:18.249240 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc0fc1-ffc4-45fb-b0f0-996c7ba645fc None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:18.249542 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc0fc1-ffc4-45fb-b0f0-996c7ba645fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:18.269369 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-fecc0fc1-ffc4-45fb-b0f0-996c7ba645fc None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:18.270657 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-fecc0fc1-ffc4-45fb-b0f0-996c7ba645fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:18.305095 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-8f382f02-4448-4766-bc25-a5ffd13e3d8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ee3e74f242748059404f8af83362497'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:18.314762 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-8f382f02-4448-4766-bc25-a5ffd13e3d8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ee3e74f242748059404f8af83362497'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:18.321569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1037/2053] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:32:17 2022] DELETE /identity/v3/users/3ee3e74f242748059404f8af83362497 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:18.326730 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f47b93-e135-4f7e-a768-d19eb48d291f None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:18.327201 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f47b93-e135-4f7e-a768-d19eb48d291f None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:18.327629 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f47b93-e135-4f7e-a768-d19eb48d291f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:18.337467 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1017/2054] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:32:18.349673 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-92f47b93-e135-4f7e-a768-d19eb48d291f None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:18.350296 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-92f47b93-e135-4f7e-a768-d19eb48d291f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:18.350874 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d0af0-dc25-46ba-aef1-cfd6b7d6616b None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:18.351278 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d0af0-dc25-46ba-aef1-cfd6b7d6616b None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:18.351563 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d0af0-dc25-46ba-aef1-cfd6b7d6616b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:18.372106 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-b56d0af0-dc25-46ba-aef1-cfd6b7d6616b None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:18.373314 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-b56d0af0-dc25-46ba-aef1-cfd6b7d6616b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:18.410275 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1038/2055] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:32:18.420472 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f47b93-e135-4f7e-a768-d19eb48d291f None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:18.421813 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-92f47b93-e135-4f7e-a768-d19eb48d291f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:18.434093 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1018/2056] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:32:18.438301 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d0af0-dc25-46ba-aef1-cfd6b7d6616b None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:18.447341 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d0af0-dc25-46ba-aef1-cfd6b7d6616b None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:18.452540 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43c7a71-0b5c-4379-9b6a-ca42300dae44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:18.453138 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c7a71-0b5c-4379-9b6a-ca42300dae44 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:18.453423 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c7a71-0b5c-4379-9b6a-ca42300dae44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:18.453700 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c7a71-0b5c-4379-9b6a-ca42300dae44 admin admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:18.454416 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43c7a71-0b5c-4379-9b6a-ca42300dae44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:32:18.454722 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43c7a71-0b5c-4379-9b6a-ca42300dae44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:18.456326 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43c7a71-0b5c-4379-9b6a-ca42300dae44 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:18.469149 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1019/2057] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:32:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:32:18.473438 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43c7a71-0b5c-4379-9b6a-ca42300dae44 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:18.479044 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f47b93-e135-4f7e-a768-d19eb48d291f None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:18.483009 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821be9b9-f106-4e29-93e1-3a8c0e6701f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:18.483468 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43c7a71-0b5c-4379-9b6a-ca42300dae44 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:18.483663 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821be9b9-f106-4e29-93e1-3a8c0e6701f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:18.483902 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821be9b9-f106-4e29-93e1-3a8c0e6701f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:18.484206 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821be9b9-f106-4e29-93e1-3a8c0e6701f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:18.488217 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7962e0c-94bb-47f3-ab57-ada95df7bdff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:18.488835 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7962e0c-94bb-47f3-ab57-ada95df7bdff admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:18.489115 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7962e0c-94bb-47f3-ab57-ada95df7bdff admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:18.489391 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7962e0c-94bb-47f3-ab57-ada95df7bdff admin admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:18.490044 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7962e0c-94bb-47f3-ab57-ada95df7bdff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:18.492252 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7962e0c-94bb-47f3-ab57-ada95df7bdff admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:18.493459 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821be9b9-f106-4e29-93e1-3a8c0e6701f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:18.494884 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821be9b9-f106-4e29-93e1-3a8c0e6701f8 service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:18.504378 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1039/2058] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:32:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 15:32:18.594740 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1020/2059] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:32:18 2022] POST /identity/v3/projects => generated 383 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:32:18.599035 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821be9b9-f106-4e29-93e1-3a8c0e6701f8 service neutron] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:18.608811 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821be9b9-f106-4e29-93e1-3a8c0e6701f8 service neutron] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:18.664094 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:18.677159 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7149a6-924e-46f1-aa8a-33c60dba705c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:18.677773 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7149a6-924e-46f1-aa8a-33c60dba705c admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:18.678058 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7149a6-924e-46f1-aa8a-33c60dba705c admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:18.678340 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7149a6-924e-46f1-aa8a-33c60dba705c admin admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:18.679014 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7149a6-924e-46f1-aa8a-33c60dba705c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:18.680984 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7149a6-924e-46f1-aa8a-33c60dba705c admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:18.774274 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1040/2060] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:32:18 2022] POST /identity/v3/users => generated 292 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:32:18.778616 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7962e0c-94bb-47f3-ab57-ada95df7bdff admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:18.790379 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7962e0c-94bb-47f3-ab57-ada95df7bdff admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:18.795342 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae0a755-42e0-41fa-98cb-b3738eb958ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:18.795962 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae0a755-42e0-41fa-98cb-b3738eb958ae admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:18.796304 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae0a755-42e0-41fa-98cb-b3738eb958ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:18.796883 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae0a755-42e0-41fa-98cb-b3738eb958ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:18.798933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae0a755-42e0-41fa-98cb-b3738eb958ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:18.800472 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae0a755-42e0-41fa-98cb-b3738eb958ae admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:18.814508 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1021/2061] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:32:18.815698 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7149a6-924e-46f1-aa8a-33c60dba705c admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:18.825806 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7149a6-924e-46f1-aa8a-33c60dba705c admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:18.831484 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c552f-a0d3-4cf6-ba6b-70a0d7c37107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:18.832242 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c552f-a0d3-4cf6-ba6b-70a0d7c37107 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:18.832530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c552f-a0d3-4cf6-ba6b-70a0d7c37107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:18.832810 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c552f-a0d3-4cf6-ba6b-70a0d7c37107 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:18.833513 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2c552f-a0d3-4cf6-ba6b-70a0d7c37107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:18.835182 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2c552f-a0d3-4cf6-ba6b-70a0d7c37107 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:18.845814 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1041/2062] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:32:18.849892 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae0a755-42e0-41fa-98cb-b3738eb958ae admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:18.859977 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae0a755-42e0-41fa-98cb-b3738eb958ae admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:18.864747 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8278736-a31c-4ab8-92d7-2dff775a953b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:18.865677 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8278736-a31c-4ab8-92d7-2dff775a953b admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:18.865963 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8278736-a31c-4ab8-92d7-2dff775a953b admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:18.866244 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8278736-a31c-4ab8-92d7-2dff775a953b admin admin] PATH_INFO: `/v3/projects/67a742a86e2a409e907d03937e468f3c/users/03b78dcbbafc44bcb6636b9f256c7597/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:18.877956 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c552f-a0d3-4cf6-ba6b-70a0d7c37107 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:18.881739 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-6f2c552f-a0d3-4cf6-ba6b-70a0d7c37107 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:18.882842 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8278736-a31c-4ab8-92d7-2dff775a953b admin admin] RBAC: Authorizing `identity:create_grant(project_id=67a742a86e2a409e907d03937e468f3c, user_id=03b78dcbbafc44bcb6636b9f256c7597, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:18.885710 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8278736-a31c-4ab8-92d7-2dff775a953b admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:18.898990 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c552f-a0d3-4cf6-ba6b-70a0d7c37107 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:18.903986 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802877c-959a-4b12-9e1e-18ecc2c34c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:18.906213 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802877c-959a-4b12-9e1e-18ecc2c34c24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:18.906498 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802877c-959a-4b12-9e1e-18ecc2c34c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:18.906791 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802877c-959a-4b12-9e1e-18ecc2c34c24 admin admin] PATH_INFO: `/v3/projects/ebcbc3b27bf04b49a224a246a7ccb5c8` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:18.908041 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0802877c-959a-4b12-9e1e-18ecc2c34c24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebcbc3b27bf04b49a224a246a7ccb5c8)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:18.909993 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0802877c-959a-4b12-9e1e-18ecc2c34c24 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:18.946456 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1022/2063] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:32:18 2022] PUT /identity/v3/projects/67a742a86e2a409e907d03937e468f3c/users/03b78dcbbafc44bcb6636b9f256c7597/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:18.957619 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee547ef4-3125-4ec7-83ce-e2d2d54bf4d9 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:18.957967 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee547ef4-3125-4ec7-83ce-e2d2d54bf4d9 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:18.958252 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee547ef4-3125-4ec7-83ce-e2d2d54bf4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:18.988514 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-ee547ef4-3125-4ec7-83ce-e2d2d54bf4d9 None None] MFA Rules not processed for user `03b78dcbbafc44bcb6636b9f256c7597`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:18.989813 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-ee547ef4-3125-4ec7-83ce-e2d2d54bf4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:18.994079 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:19.030587 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-0802877c-959a-4b12-9e1e-18ecc2c34c24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebcbc3b27bf04b49a224a246a7ccb5c8', 'request_id': 'req-0802877c-959a-4b12-9e1e-18ecc2c34c24'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:19.041581 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-0802877c-959a-4b12-9e1e-18ecc2c34c24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebcbc3b27bf04b49a224a246a7ccb5c8', 'request_id': 'req-0802877c-959a-4b12-9e1e-18ecc2c34c24'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:19.053353 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-0802877c-959a-4b12-9e1e-18ecc2c34c24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebcbc3b27bf04b49a224a246a7ccb5c8', 'request_id': 'req-0802877c-959a-4b12-9e1e-18ecc2c34c24'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:19.067238 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1023/2064] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:32:19.071793 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee547ef4-3125-4ec7-83ce-e2d2d54bf4d9 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:19.121224 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-0802877c-959a-4b12-9e1e-18ecc2c34c24 admin admin] The token cache is being invalidate because project ebcbc3b27bf04b49a224a246a7ccb5c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:32:19.121516 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-0802877c-959a-4b12-9e1e-18ecc2c34c24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:19.122177 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-0802877c-959a-4b12-9e1e-18ecc2c34c24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:19.124436 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1042/2065] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:32:18 2022] DELETE /identity/v3/projects/ebcbc3b27bf04b49a224a246a7ccb5c8 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:32:19.128476 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802877c-959a-4b12-9e1e-18ecc2c34c24 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:19.129764 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-0802877c-959a-4b12-9e1e-18ecc2c34c24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:19.141534 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee547ef4-3125-4ec7-83ce-e2d2d54bf4d9 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:19.202418 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802877c-959a-4b12-9e1e-18ecc2c34c24 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:19.206288 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92fd578-bd41-4a67-944c-e67af75560f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:19.206922 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92fd578-bd41-4a67-944c-e67af75560f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:19.207155 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01e1292-6bd7-487e-bbda-77fbc15a0201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:19.207271 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92fd578-bd41-4a67-944c-e67af75560f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:19.207469 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92fd578-bd41-4a67-944c-e67af75560f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:19.207794 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e1292-6bd7-487e-bbda-77fbc15a0201 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:19.208106 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e1292-6bd7-487e-bbda-77fbc15a0201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:19.208203 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92fd578-bd41-4a67-944c-e67af75560f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:19.208450 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e1292-6bd7-487e-bbda-77fbc15a0201 admin admin] PATH_INFO: `/v3/users/3b3e2822d0164644b12a0c606b7b848c` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:19.209737 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01e1292-6bd7-487e-bbda-77fbc15a0201 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b3e2822d0164644b12a0c606b7b848c)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:19.210100 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92fd578-bd41-4a67-944c-e67af75560f4 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:19.211730 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01e1292-6bd7-487e-bbda-77fbc15a0201 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:19.311205 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1024/2066] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:32:19 2022] POST /identity/v3/projects => generated 381 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:32:19.315096 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92fd578-bd41-4a67-944c-e67af75560f4 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:19.316759 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-d92fd578-bd41-4a67-944c-e67af75560f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:19.382922 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92fd578-bd41-4a67-944c-e67af75560f4 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:19.387661 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d3e85-1aa5-44fb-b3b7-79e90933b936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:19.388316 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d3e85-1aa5-44fb-b3b7-79e90933b936 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:19.388623 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d3e85-1aa5-44fb-b3b7-79e90933b936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:19.388890 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d3e85-1aa5-44fb-b3b7-79e90933b936 admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:19.389864 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d3e85-1aa5-44fb-b3b7-79e90933b936 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:19.392327 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d3e85-1aa5-44fb-b3b7-79e90933b936 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:19.433380 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-b01e1292-6bd7-487e-bbda-77fbc15a0201 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b3e2822d0164644b12a0c606b7b848c'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:19.434036 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-b01e1292-6bd7-487e-bbda-77fbc15a0201 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b3e2822d0164644b12a0c606b7b848c'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:19.436006 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=82,keystone:INSERT=4,keystone:DELETE=14 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:19.446604 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-b01e1292-6bd7-487e-bbda-77fbc15a0201 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b3e2822d0164644b12a0c606b7b848c'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:19.493896 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1025/2067] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:32:19 2022] POST /identity/v3/users => generated 297 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:32:19.497706 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d3e85-1aa5-44fb-b3b7-79e90933b936 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:19.499293 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-0f4d3e85-1aa5-44fb-b3b7-79e90933b936 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:19.511325 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d3e85-1aa5-44fb-b3b7-79e90933b936 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:19.516090 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c559cc-8137-48b6-b2d8-3a223df572ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:19.516665 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c559cc-8137-48b6-b2d8-3a223df572ce admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:19.516950 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c559cc-8137-48b6-b2d8-3a223df572ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:19.517229 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c559cc-8137-48b6-b2d8-3a223df572ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:19.517911 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c559cc-8137-48b6-b2d8-3a223df572ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:19.519481 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c559cc-8137-48b6-b2d8-3a223df572ce admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:19.522248 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-b01e1292-6bd7-487e-bbda-77fbc15a0201 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b3e2822d0164644b12a0c606b7b848c'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:19.530836 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-b01e1292-6bd7-487e-bbda-77fbc15a0201 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b3e2822d0164644b12a0c606b7b848c'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:19.532071 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1026/2068] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 15:32:19.536183 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c559cc-8137-48b6-b2d8-3a223df572ce admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:19.537075 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1043/2069] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:32:19 2022] DELETE /identity/v3/users/3b3e2822d0164644b12a0c606b7b848c => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:19.546840 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c559cc-8137-48b6-b2d8-3a223df572ce admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:19.580716 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:19.604299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f265b1-9e96-4304-857e-61371e4e9eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:19.604973 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f265b1-9e96-4304-857e-61371e4e9eae admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:19.605259 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f265b1-9e96-4304-857e-61371e4e9eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:19.608234 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f265b1-9e96-4304-857e-61371e4e9eae admin admin] PATH_INFO: `/v3/projects/34e7683fe6a04987b177adec4a4f435a/users/c09ec63036ee44ec8ed70a7baad8ebae/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:19.622355 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f265b1-9e96-4304-857e-61371e4e9eae admin admin] RBAC: Authorizing `identity:create_grant(project_id=34e7683fe6a04987b177adec4a4f435a, user_id=c09ec63036ee44ec8ed70a7baad8ebae, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:19.625122 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f265b1-9e96-4304-857e-61371e4e9eae admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:19.682385 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1027/2070] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:32:19 2022] PUT /identity/v3/projects/34e7683fe6a04987b177adec4a4f435a/users/c09ec63036ee44ec8ed70a7baad8ebae/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:19.686866 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01e1292-6bd7-487e-bbda-77fbc15a0201 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:19.697142 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01e1292-6bd7-487e-bbda-77fbc15a0201 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:19.761752 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32fe18-936a-4868-b6b7-c14ca3e8b9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:19.762420 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32fe18-936a-4868-b6b7-c14ca3e8b9af admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:19.762708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32fe18-936a-4868-b6b7-c14ca3e8b9af admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:19.762990 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32fe18-936a-4868-b6b7-c14ca3e8b9af admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:19.765617 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32fe18-936a-4868-b6b7-c14ca3e8b9af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:19.766112 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32fe18-936a-4868-b6b7-c14ca3e8b9af admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:19.776375 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1044/2071] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 15:32:19.780968 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f265b1-9e96-4304-857e-61371e4e9eae admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:19.792343 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f265b1-9e96-4304-857e-61371e4e9eae admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:19.797240 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50eab9c-5d8d-4e50-908e-0034d7d04924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:19.797907 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50eab9c-5d8d-4e50-908e-0034d7d04924 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:19.798188 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50eab9c-5d8d-4e50-908e-0034d7d04924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:19.798815 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50eab9c-5d8d-4e50-908e-0034d7d04924 admin admin] PATH_INFO: `/v3/projects/34e7683fe6a04987b177adec4a4f435a/users/c09ec63036ee44ec8ed70a7baad8ebae/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:19.800845 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50eab9c-5d8d-4e50-908e-0034d7d04924 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34e7683fe6a04987b177adec4a4f435a, user_id=c09ec63036ee44ec8ed70a7baad8ebae, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:19.803495 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50eab9c-5d8d-4e50-908e-0034d7d04924 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:19.814059 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1028/2072] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:32:19 2022] PUT /identity/v3/projects/34e7683fe6a04987b177adec4a4f435a/users/c09ec63036ee44ec8ed70a7baad8ebae/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:19.825329 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b4fb2-a562-4a28-8b78-acf2aed70148 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:19.825817 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b4fb2-a562-4a28-8b78-acf2aed70148 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:19.826667 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b4fb2-a562-4a28-8b78-acf2aed70148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:19.857151 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-c95b4fb2-a562-4a28-8b78-acf2aed70148 None None] MFA Rules not processed for user `c09ec63036ee44ec8ed70a7baad8ebae`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:19.858568 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-c95b4fb2-a562-4a28-8b78-acf2aed70148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:19.926230 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1045/2073] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:19 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:19.937504 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50eab9c-5d8d-4e50-908e-0034d7d04924 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:19.939805 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-b50eab9c-5d8d-4e50-908e-0034d7d04924 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:19.964368 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95b4fb2-a562-4a28-8b78-acf2aed70148 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:19.966266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-c95b4fb2-a562-4a28-8b78-acf2aed70148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:20.009149 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50eab9c-5d8d-4e50-908e-0034d7d04924 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:20.013042 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c1641-f098-40c7-8b81-568b94708593 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:20.014180 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c1641-f098-40c7-8b81-568b94708593 service cinder] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:20.014467 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c1641-f098-40c7-8b81-568b94708593 service cinder] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:20.014742 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c1641-f098-40c7-8b81-568b94708593 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:20.025522 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c1641-f098-40c7-8b81-568b94708593 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:20.027025 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c1641-f098-40c7-8b81-568b94708593 service cinder] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:20.038488 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1029/2074] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:32:19 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:32:20.045733 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:20.063667 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95b4fb2-a562-4a28-8b78-acf2aed70148 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:20.071069 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a163537-9728-4ee2-975f-f6d11993f2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:20.071609 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a163537-9728-4ee2-975f-f6d11993f2f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:20.071852 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a163537-9728-4ee2-975f-f6d11993f2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:20.072161 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a163537-9728-4ee2-975f-f6d11993f2f4 admin admin] PATH_INFO: `/v3/projects/7e75583df48248818e8c19308152fdbc` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:20.073378 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a163537-9728-4ee2-975f-f6d11993f2f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e75583df48248818e8c19308152fdbc)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:20.075280 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a163537-9728-4ee2-975f-f6d11993f2f4 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:20.168113 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c1641-f098-40c7-8b81-568b94708593 service cinder] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:20.174061 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c1641-f098-40c7-8b81-568b94708593 service cinder] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:20.178559 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90496e1e-1f23-4d1a-b09d-e64a2e95f3d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:20.179182 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90496e1e-1f23-4d1a-b09d-e64a2e95f3d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:20.179460 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90496e1e-1f23-4d1a-b09d-e64a2e95f3d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:20.179736 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90496e1e-1f23-4d1a-b09d-e64a2e95f3d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:20.182495 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-90496e1e-1f23-4d1a-b09d-e64a2e95f3d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:20.197487 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-1a163537-9728-4ee2-975f-f6d11993f2f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e75583df48248818e8c19308152fdbc', 'request_id': 'req-1a163537-9728-4ee2-975f-f6d11993f2f4'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:20.207472 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-1a163537-9728-4ee2-975f-f6d11993f2f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e75583df48248818e8c19308152fdbc', 'request_id': 'req-1a163537-9728-4ee2-975f-f6d11993f2f4'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:20.214303 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-1a163537-9728-4ee2-975f-f6d11993f2f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e75583df48248818e8c19308152fdbc', 'request_id': 'req-1a163537-9728-4ee2-975f-f6d11993f2f4'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:20.220881 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:20.273701 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90496e1e-1f23-4d1a-b09d-e64a2e95f3d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:20.284674 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90496e1e-1f23-4d1a-b09d-e64a2e95f3d7 service cinder] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:20.303081 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-1a163537-9728-4ee2-975f-f6d11993f2f4 admin admin] The token cache is being invalidate because project 7e75583df48248818e8c19308152fdbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:32:20.303735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-1a163537-9728-4ee2-975f-f6d11993f2f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:20.304709 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-1a163537-9728-4ee2-975f-f6d11993f2f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:20.305607 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1030/2075] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:32:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 15:32:20.306538 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1046/2076] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:32:19 2022] DELETE /identity/v3/projects/7e75583df48248818e8c19308152fdbc => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:32:20.313593 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a22197-8f57-4e4b-9740-a62f7f216ece None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:20.314061 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a22197-8f57-4e4b-9740-a62f7f216ece None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:20.314482 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a22197-8f57-4e4b-9740-a62f7f216ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:20.348228 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-b0a22197-8f57-4e4b-9740-a62f7f216ece None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:20.349236 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-b0a22197-8f57-4e4b-9740-a62f7f216ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:20.420135 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1031/2077] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:20.428135 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c5f33-4ccb-4fee-bca6-4c6df84d16fa None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:20.428436 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c5f33-4ccb-4fee-bca6-4c6df84d16fa None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:20.428724 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c5f33-4ccb-4fee-bca6-4c6df84d16fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:20.450878 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-4f8c5f33-4ccb-4fee-bca6-4c6df84d16fa None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:20.452732 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-4f8c5f33-4ccb-4fee-bca6-4c6df84d16fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:20.462230 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1047/2078] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:20.466380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a22197-8f57-4e4b-9740-a62f7f216ece None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:20.476324 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a22197-8f57-4e4b-9740-a62f7f216ece None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:20.481044 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd903652-6975-473a-b3f5-2e410fe92df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:20.481656 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd903652-6975-473a-b3f5-2e410fe92df2 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:20.481937 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd903652-6975-473a-b3f5-2e410fe92df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:20.482210 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd903652-6975-473a-b3f5-2e410fe92df2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:20.483421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd903652-6975-473a-b3f5-2e410fe92df2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:32:20.483728 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd903652-6975-473a-b3f5-2e410fe92df2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:20.485352 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd903652-6975-473a-b3f5-2e410fe92df2 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:20.497136 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1032/2079] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:32:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:32:20.502736 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c5f33-4ccb-4fee-bca6-4c6df84d16fa None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:20.512375 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c5f33-4ccb-4fee-bca6-4c6df84d16fa None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:20.517179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d1dfd9-54a6-4d9e-9573-033650c1596b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:20.517790 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1dfd9-54a6-4d9e-9573-033650c1596b admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:20.518077 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1dfd9-54a6-4d9e-9573-033650c1596b admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:20.518359 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1dfd9-54a6-4d9e-9573-033650c1596b admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:20.519062 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d1dfd9-54a6-4d9e-9573-033650c1596b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:20.521021 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d1dfd9-54a6-4d9e-9573-033650c1596b admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:20.642060 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1048/2080] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:32:20 2022] POST /identity/v3/projects => generated 409 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:32:20.643066 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd903652-6975-473a-b3f5-2e410fe92df2 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:20.659831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd903652-6975-473a-b3f5-2e410fe92df2 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:20.717106 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54424d6e-358f-4502-baae-0ff8ced2511f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:20.717873 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54424d6e-358f-4502-baae-0ff8ced2511f admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:20.718167 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54424d6e-358f-4502-baae-0ff8ced2511f admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:20.718445 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54424d6e-358f-4502-baae-0ff8ced2511f admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:20.719115 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54424d6e-358f-4502-baae-0ff8ced2511f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:20.721108 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54424d6e-358f-4502-baae-0ff8ced2511f admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:20.797300 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1033/2081] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:32:20 2022] POST /identity/v3/users => generated 305 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:32:20.801492 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d1dfd9-54a6-4d9e-9573-033650c1596b admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:20.812460 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d1dfd9-54a6-4d9e-9573-033650c1596b admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:20.817177 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41867d39-05e1-4f2b-923a-b732e087d55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:20.817794 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41867d39-05e1-4f2b-923a-b732e087d55e admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:20.818078 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41867d39-05e1-4f2b-923a-b732e087d55e admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:20.818357 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41867d39-05e1-4f2b-923a-b732e087d55e admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:20.819054 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41867d39-05e1-4f2b-923a-b732e087d55e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:20.820726 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41867d39-05e1-4f2b-923a-b732e087d55e admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:20.834629 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1049/2082] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:32:20.836512 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54424d6e-358f-4502-baae-0ff8ced2511f admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:20.848229 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54424d6e-358f-4502-baae-0ff8ced2511f admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:20.853566 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2206d5f2-bbd6-45dc-97b3-34c3b431b65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:20.854213 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2206d5f2-bbd6-45dc-97b3-34c3b431b65a admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:20.854497 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2206d5f2-bbd6-45dc-97b3-34c3b431b65a admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:20.854816 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2206d5f2-bbd6-45dc-97b3-34c3b431b65a admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:20.855499 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2206d5f2-bbd6-45dc-97b3-34c3b431b65a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:20.857123 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2206d5f2-bbd6-45dc-97b3-34c3b431b65a admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:20.879216 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1034/2083] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:32:20.880053 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41867d39-05e1-4f2b-923a-b732e087d55e admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:20.891186 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41867d39-05e1-4f2b-923a-b732e087d55e admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:20.896290 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606a3071-7492-4234-9880-16cf7a15ad09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:20.897012 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a3071-7492-4234-9880-16cf7a15ad09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:20.897297 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a3071-7492-4234-9880-16cf7a15ad09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:20.897574 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a3071-7492-4234-9880-16cf7a15ad09 admin admin] PATH_INFO: `/v3/projects/4780c1a68871410494f8e5631eab73df/users/4e6072ef3e1f435f86e3a0babc2903f9/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:20.931718 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606a3071-7492-4234-9880-16cf7a15ad09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4780c1a68871410494f8e5631eab73df, user_id=4e6072ef3e1f435f86e3a0babc2903f9, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:20.954190 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606a3071-7492-4234-9880-16cf7a15ad09 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:21.039705 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1050/2084] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:32:20 2022] PUT /identity/v3/projects/4780c1a68871410494f8e5631eab73df/users/4e6072ef3e1f435f86e3a0babc2903f9/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:32:21.050783 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a58afd-5001-4675-831e-d18ceb6a0033 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:21.051162 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a58afd-5001-4675-831e-d18ceb6a0033 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:21.051514 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a58afd-5001-4675-831e-d18ceb6a0033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:21.084663 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-45a58afd-5001-4675-831e-d18ceb6a0033 None None] MFA Rules not processed for user `4e6072ef3e1f435f86e3a0babc2903f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:21.086959 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-45a58afd-5001-4675-831e-d18ceb6a0033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:21.108034 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:21.152558 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1035/2085] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:21 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:32:21.157623 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606a3071-7492-4234-9880-16cf7a15ad09 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:21.222955 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606a3071-7492-4234-9880-16cf7a15ad09 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:21.227895 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdebff63-71e5-4267-8fbc-258ab3fa4757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:21.228556 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdebff63-71e5-4267-8fbc-258ab3fa4757 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:21.228846 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdebff63-71e5-4267-8fbc-258ab3fa4757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:21.229181 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdebff63-71e5-4267-8fbc-258ab3fa4757 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:21.229894 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdebff63-71e5-4267-8fbc-258ab3fa4757 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:21.231876 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdebff63-71e5-4267-8fbc-258ab3fa4757 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:21.346277 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:21.352834 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1051/2086] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:32:21 2022] POST /identity/v3/projects => generated 409 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:32:21.356930 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a58afd-5001-4675-831e-d18ceb6a0033 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:21.368031 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a58afd-5001-4675-831e-d18ceb6a0033 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:21.426941 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197b073d-b754-4659-91b8-11c59f542ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:21.427660 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b073d-b754-4659-91b8-11c59f542ae2 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:21.428153 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b073d-b754-4659-91b8-11c59f542ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:21.428503 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b073d-b754-4659-91b8-11c59f542ae2 admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:21.429231 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197b073d-b754-4659-91b8-11c59f542ae2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:21.431208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197b073d-b754-4659-91b8-11c59f542ae2 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:21.502381 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1036/2087] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:32:21 2022] POST /identity/v3/users => generated 311 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:32:21.503720 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdebff63-71e5-4267-8fbc-258ab3fa4757 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:21.515492 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdebff63-71e5-4267-8fbc-258ab3fa4757 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:21.521189 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe81393c-bc9d-489c-8eef-705d4dea4269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:21.521823 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe81393c-bc9d-489c-8eef-705d4dea4269 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:21.522110 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe81393c-bc9d-489c-8eef-705d4dea4269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:21.522914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe81393c-bc9d-489c-8eef-705d4dea4269 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:21.524385 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe81393c-bc9d-489c-8eef-705d4dea4269 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:21.525464 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe81393c-bc9d-489c-8eef-705d4dea4269 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:21.537628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1052/2088] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:32:21.541810 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197b073d-b754-4659-91b8-11c59f542ae2 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:21.552175 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197b073d-b754-4659-91b8-11c59f542ae2 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:21.556947 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9f0e8d-0891-45f7-8d0a-882da597f4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:21.557651 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f0e8d-0891-45f7-8d0a-882da597f4bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:21.557940 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f0e8d-0891-45f7-8d0a-882da597f4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:21.558215 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f0e8d-0891-45f7-8d0a-882da597f4bf admin admin] PATH_INFO: `/v3/projects/e076428ec4f2421693186611b1789c0b/users/4562496d43b64fb3b2249a8c0148da72/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:21.574396 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9f0e8d-0891-45f7-8d0a-882da597f4bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e076428ec4f2421693186611b1789c0b, user_id=4562496d43b64fb3b2249a8c0148da72, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:21.577717 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9f0e8d-0891-45f7-8d0a-882da597f4bf admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:21.654177 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1037/2089] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:32:21 2022] PUT /identity/v3/projects/e076428ec4f2421693186611b1789c0b/users/4562496d43b64fb3b2249a8c0148da72/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:21.658069 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe81393c-bc9d-489c-8eef-705d4dea4269 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:21.668453 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe81393c-bc9d-489c-8eef-705d4dea4269 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:21.728371 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9076755-bd2e-4909-ba5b-6177825b3b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:21.728952 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9076755-bd2e-4909-ba5b-6177825b3b60 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:21.729245 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9076755-bd2e-4909-ba5b-6177825b3b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:21.729531 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9076755-bd2e-4909-ba5b-6177825b3b60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:21.730303 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9076755-bd2e-4909-ba5b-6177825b3b60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:21.731930 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9076755-bd2e-4909-ba5b-6177825b3b60 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:21.742394 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1053/2090] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:32:21.746482 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9f0e8d-0891-45f7-8d0a-882da597f4bf admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:21.757577 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9f0e8d-0891-45f7-8d0a-882da597f4bf admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:21.762270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fcac0d-c845-4b8a-8219-f11879fee2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:21.762930 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fcac0d-c845-4b8a-8219-f11879fee2da admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:21.763216 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fcac0d-c845-4b8a-8219-f11879fee2da admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:21.763497 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fcac0d-c845-4b8a-8219-f11879fee2da admin admin] PATH_INFO: `/v3/projects/e076428ec4f2421693186611b1789c0b/users/4562496d43b64fb3b2249a8c0148da72/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:21.766129 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fcac0d-c845-4b8a-8219-f11879fee2da admin admin] RBAC: Authorizing `identity:create_grant(project_id=e076428ec4f2421693186611b1789c0b, user_id=4562496d43b64fb3b2249a8c0148da72, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:21.768728 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fcac0d-c845-4b8a-8219-f11879fee2da admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:21.778990 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1038/2091] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:32:21 2022] PUT /identity/v3/projects/e076428ec4f2421693186611b1789c0b/users/4562496d43b64fb3b2249a8c0148da72/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:32:21.790598 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d0a24-dfe9-434e-b2bd-06b86c13d7a5 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:21.790889 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d0a24-dfe9-434e-b2bd-06b86c13d7a5 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:21.791173 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d0a24-dfe9-434e-b2bd-06b86c13d7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:21.820123 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-df4d0a24-dfe9-434e-b2bd-06b86c13d7a5 None None] MFA Rules not processed for user `4562496d43b64fb3b2249a8c0148da72`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:21.821345 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-df4d0a24-dfe9-434e-b2bd-06b86c13d7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:21.888880 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1054/2092] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:21 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:21.900277 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fcac0d-c845-4b8a-8219-f11879fee2da admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:21.902001 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-e1fcac0d-c845-4b8a-8219-f11879fee2da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:21.962848 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fcac0d-c845-4b8a-8219-f11879fee2da admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:21.966517 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc2fd39-ae76-4a16-920e-3ce4dcb963d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:21.967127 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2fd39-ae76-4a16-920e-3ce4dcb963d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:21.967406 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2fd39-ae76-4a16-920e-3ce4dcb963d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:21.967847 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2fd39-ae76-4a16-920e-3ce4dcb963d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:21.977133 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc2fd39-ae76-4a16-920e-3ce4dcb963d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:21.978646 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc2fd39-ae76-4a16-920e-3ce4dcb963d2 service cinder] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:21.988456 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1039/2093] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:32:21 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:32:22.566094 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d0a24-dfe9-434e-b2bd-06b86c13d7a5 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:22.575706 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d0a24-dfe9-434e-b2bd-06b86c13d7a5 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:22.579356 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e7192e-0b0d-40f5-af9d-fe7e2df1d565 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:22.579963 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7192e-0b0d-40f5-af9d-fe7e2df1d565 service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:22.580431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7192e-0b0d-40f5-af9d-fe7e2df1d565 service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:22.580708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7192e-0b0d-40f5-af9d-fe7e2df1d565 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:22.640637 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e7192e-0b0d-40f5-af9d-fe7e2df1d565 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:22.642104 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e7192e-0b0d-40f5-af9d-fe7e2df1d565 service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:22.652342 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1055/2094] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:32:22 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:32:24.398383 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc2fd39-ae76-4a16-920e-3ce4dcb963d2 service cinder] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:24.414714 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-afc2fd39-ae76-4a16-920e-3ce4dcb963d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:24.445625 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:24.521409 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc2fd39-ae76-4a16-920e-3ce4dcb963d2 service cinder] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:24.526909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f23b99-bf3b-4f7f-9ca7-02b1447bdf4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:24.527664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f23b99-bf3b-4f7f-9ca7-02b1447bdf4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:24.528091 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f23b99-bf3b-4f7f-9ca7-02b1447bdf4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:24.528488 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f23b99-bf3b-4f7f-9ca7-02b1447bdf4e admin admin] PATH_INFO: `/v3/users/03b78dcbbafc44bcb6636b9f256c7597` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:24.530458 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f23b99-bf3b-4f7f-9ca7-02b1447bdf4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=03b78dcbbafc44bcb6636b9f256c7597)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:24.532502 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f23b99-bf3b-4f7f-9ca7-02b1447bdf4e admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:24.734814 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-a9f23b99-bf3b-4f7f-9ca7-02b1447bdf4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03b78dcbbafc44bcb6636b9f256c7597'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:24.742102 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-a9f23b99-bf3b-4f7f-9ca7-02b1447bdf4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03b78dcbbafc44bcb6636b9f256c7597'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:24.742807 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-a9f23b99-bf3b-4f7f-9ca7-02b1447bdf4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03b78dcbbafc44bcb6636b9f256c7597'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:24.759799 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-a9f23b99-bf3b-4f7f-9ca7-02b1447bdf4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03b78dcbbafc44bcb6636b9f256c7597'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:24.759799 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-a9f23b99-bf3b-4f7f-9ca7-02b1447bdf4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03b78dcbbafc44bcb6636b9f256c7597'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:24.853840 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1040/2095] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:32:24 2022] DELETE /identity/v3/users/03b78dcbbafc44bcb6636b9f256c7597 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:24.858479 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea959f3-7d98-49f8-b41d-fa7037a2ea58 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:24.859069 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea959f3-7d98-49f8-b41d-fa7037a2ea58 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:24.859306 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea959f3-7d98-49f8-b41d-fa7037a2ea58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:24.879876 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-5ea959f3-7d98-49f8-b41d-fa7037a2ea58 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:24.881136 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-5ea959f3-7d98-49f8-b41d-fa7037a2ea58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:24.885169 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:24.954579 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1056/2096] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:32:24.965295 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f23b99-bf3b-4f7f-9ca7-02b1447bdf4e admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:24.966670 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-a9f23b99-bf3b-4f7f-9ca7-02b1447bdf4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:25.032056 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f23b99-bf3b-4f7f-9ca7-02b1447bdf4e admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:25.035706 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f6d856-6fe6-49e7-ae89-6eac0255896b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:25.036355 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6d856-6fe6-49e7-ae89-6eac0255896b service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:25.036639 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6d856-6fe6-49e7-ae89-6eac0255896b service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:25.036912 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6d856-6fe6-49e7-ae89-6eac0255896b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:25.046460 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f6d856-6fe6-49e7-ae89-6eac0255896b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:25.047871 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f6d856-6fe6-49e7-ae89-6eac0255896b service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:25.057677 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1041/2097] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:32:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:32:25.114786 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea959f3-7d98-49f8-b41d-fa7037a2ea58 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:25.116608 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-5ea959f3-7d98-49f8-b41d-fa7037a2ea58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:25.129149 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea959f3-7d98-49f8-b41d-fa7037a2ea58 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:25.133775 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a5b14-98ed-4ea3-a891-8aee8eb644fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:25.134417 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a5b14-98ed-4ea3-a891-8aee8eb644fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:25.134702 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a5b14-98ed-4ea3-a891-8aee8eb644fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:25.134980 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a5b14-98ed-4ea3-a891-8aee8eb644fa admin admin] PATH_INFO: `/v3/users/0607e0b0b98843da830fc85e100fcf2a` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:25.136275 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a5b14-98ed-4ea3-a891-8aee8eb644fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=0607e0b0b98843da830fc85e100fcf2a)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:25.138271 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a5b14-98ed-4ea3-a891-8aee8eb644fa admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:25.318258 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-8b6a5b14-98ed-4ea3-a891-8aee8eb644fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0607e0b0b98843da830fc85e100fcf2a'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:25.318965 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-8b6a5b14-98ed-4ea3-a891-8aee8eb644fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0607e0b0b98843da830fc85e100fcf2a'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:25.332990 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-8b6a5b14-98ed-4ea3-a891-8aee8eb644fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0607e0b0b98843da830fc85e100fcf2a'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:25.401376 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-8b6a5b14-98ed-4ea3-a891-8aee8eb644fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0607e0b0b98843da830fc85e100fcf2a'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:25.409552 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-8b6a5b14-98ed-4ea3-a891-8aee8eb644fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0607e0b0b98843da830fc85e100fcf2a'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:25.415689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1057/2098] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:32:25 2022] DELETE /identity/v3/users/0607e0b0b98843da830fc85e100fcf2a => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:25.420244 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f1670-aa37-435f-8640-3a738315db95 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:25.421299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f1670-aa37-435f-8640-3a738315db95 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:25.421488 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f1670-aa37-435f-8640-3a738315db95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:25.440188 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-e16f1670-aa37-435f-8640-3a738315db95 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:25.441407 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-e16f1670-aa37-435f-8640-3a738315db95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:25.472277 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a5b14-98ed-4ea3-a891-8aee8eb644fa admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:25.473991 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-8b6a5b14-98ed-4ea3-a891-8aee8eb644fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:25.476159 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:25.505756 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1042/2099] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:25.519893 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f1670-aa37-435f-8640-3a738315db95 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:25.522969 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-e16f1670-aa37-435f-8640-3a738315db95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:25.577278 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a5b14-98ed-4ea3-a891-8aee8eb644fa admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:25.580404 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f1670-aa37-435f-8640-3a738315db95 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:25.582316 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f3696-dba9-4970-8a68-d6662b972522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:25.583032 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f3696-dba9-4970-8a68-d6662b972522 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:25.583315 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f3696-dba9-4970-8a68-d6662b972522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:25.583593 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f3696-dba9-4970-8a68-d6662b972522 admin admin] PATH_INFO: `/v3/projects/67a742a86e2a409e907d03937e468f3c` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:25.584183 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e5f4a1-964c-4cd2-8e90-3d487b2ba7ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:25.584769 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5f4a1-964c-4cd2-8e90-3d487b2ba7ce service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:25.584875 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130f3696-dba9-4970-8a68-d6662b972522 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67a742a86e2a409e907d03937e468f3c)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:25.585062 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5f4a1-964c-4cd2-8e90-3d487b2ba7ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:25.585345 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5f4a1-964c-4cd2-8e90-3d487b2ba7ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:25.586845 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130f3696-dba9-4970-8a68-d6662b972522 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:25.595169 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e5f4a1-964c-4cd2-8e90-3d487b2ba7ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:25.596643 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e5f4a1-964c-4cd2-8e90-3d487b2ba7ce service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:25.606150 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1043/2100] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:32:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:32:25.614190 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:25.722899 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-130f3696-dba9-4970-8a68-d6662b972522 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67a742a86e2a409e907d03937e468f3c', 'request_id': 'req-130f3696-dba9-4970-8a68-d6662b972522'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:25.729030 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-130f3696-dba9-4970-8a68-d6662b972522 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67a742a86e2a409e907d03937e468f3c', 'request_id': 'req-130f3696-dba9-4970-8a68-d6662b972522'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:25.734617 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-130f3696-dba9-4970-8a68-d6662b972522 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67a742a86e2a409e907d03937e468f3c', 'request_id': 'req-130f3696-dba9-4970-8a68-d6662b972522'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:25.818805 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-130f3696-dba9-4970-8a68-d6662b972522 admin admin] The token cache is being invalidate because project 67a742a86e2a409e907d03937e468f3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:32:25.819098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-130f3696-dba9-4970-8a68-d6662b972522 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:25.819794 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-130f3696-dba9-4970-8a68-d6662b972522 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:25.821577 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1058/2101] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:32:25 2022] DELETE /identity/v3/projects/67a742a86e2a409e907d03937e468f3c => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:32:25.825448 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e5f4a1-964c-4cd2-8e90-3d487b2ba7ce service neutron] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:25.838029 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e5f4a1-964c-4cd2-8e90-3d487b2ba7ce service neutron] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:25.899431 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b6280-ef2b-484b-81df-3752b2b99e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:25.900175 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b6280-ef2b-484b-81df-3752b2b99e17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:25.900458 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b6280-ef2b-484b-81df-3752b2b99e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:25.900737 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b6280-ef2b-484b-81df-3752b2b99e17 admin admin] PATH_INFO: `/v3/users/c09ec63036ee44ec8ed70a7baad8ebae` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:25.901903 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b6280-ef2b-484b-81df-3752b2b99e17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c09ec63036ee44ec8ed70a7baad8ebae)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:25.903902 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b6280-ef2b-484b-81df-3752b2b99e17 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:26.072411 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f3696-dba9-4970-8a68-d6662b972522 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:26.073846 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-130f3696-dba9-4970-8a68-d6662b972522 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:26.090127 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f3696-dba9-4970-8a68-d6662b972522 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:26.094901 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13deaccb-0d42-403f-b446-6f3a11e25725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:26.095597 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13deaccb-0d42-403f-b446-6f3a11e25725 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:26.095903 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13deaccb-0d42-403f-b446-6f3a11e25725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:26.096222 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13deaccb-0d42-403f-b446-6f3a11e25725 admin admin] PATH_INFO: `/v3/projects/fac601f1d2954638a3a70e137048a69a` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:26.097419 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13deaccb-0d42-403f-b446-6f3a11e25725 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fac601f1d2954638a3a70e137048a69a)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:26.107223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13deaccb-0d42-403f-b446-6f3a11e25725 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:26.198913 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-b36b6280-ef2b-484b-81df-3752b2b99e17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c09ec63036ee44ec8ed70a7baad8ebae'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:26.204288 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-13deaccb-0d42-403f-b446-6f3a11e25725 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fac601f1d2954638a3a70e137048a69a', 'request_id': 'req-13deaccb-0d42-403f-b446-6f3a11e25725'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:26.206477 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-b36b6280-ef2b-484b-81df-3752b2b99e17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c09ec63036ee44ec8ed70a7baad8ebae'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:26.207146 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-b36b6280-ef2b-484b-81df-3752b2b99e17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c09ec63036ee44ec8ed70a7baad8ebae'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:26.210869 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-13deaccb-0d42-403f-b446-6f3a11e25725 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fac601f1d2954638a3a70e137048a69a', 'request_id': 'req-13deaccb-0d42-403f-b446-6f3a11e25725'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:26.216705 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-13deaccb-0d42-403f-b446-6f3a11e25725 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fac601f1d2954638a3a70e137048a69a', 'request_id': 'req-13deaccb-0d42-403f-b446-6f3a11e25725'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:26.220227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-b36b6280-ef2b-484b-81df-3752b2b99e17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c09ec63036ee44ec8ed70a7baad8ebae'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:26.220898 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-b36b6280-ef2b-484b-81df-3752b2b99e17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c09ec63036ee44ec8ed70a7baad8ebae'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:26.303906 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-13deaccb-0d42-403f-b446-6f3a11e25725 admin admin] The token cache is being invalidate because project fac601f1d2954638a3a70e137048a69a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:32:26.304265 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-13deaccb-0d42-403f-b446-6f3a11e25725 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:26.304883 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-13deaccb-0d42-403f-b446-6f3a11e25725 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:26.306399 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1059/2102] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:32:26 2022] DELETE /identity/v3/projects/fac601f1d2954638a3a70e137048a69a => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:26.310253 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13deaccb-0d42-403f-b446-6f3a11e25725 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:26.311524 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-13deaccb-0d42-403f-b446-6f3a11e25725 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:26.356160 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1044/2103] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:32:25 2022] DELETE /identity/v3/users/c09ec63036ee44ec8ed70a7baad8ebae => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:26.378825 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13deaccb-0d42-403f-b446-6f3a11e25725 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:26.437831 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:26.458183 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e1505a-3477-48d5-8287-d20557f5083e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:26.458851 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e1505a-3477-48d5-8287-d20557f5083e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:26.459127 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e1505a-3477-48d5-8287-d20557f5083e admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:26.459408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e1505a-3477-48d5-8287-d20557f5083e admin admin] PATH_INFO: `/v3/users/b463e71faa6b4c889b8cbbaecdac42c1` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:26.461141 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e1505a-3477-48d5-8287-d20557f5083e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b463e71faa6b4c889b8cbbaecdac42c1)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:26.463320 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e1505a-3477-48d5-8287-d20557f5083e admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:26.663519 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c3e1505a-3477-48d5-8287-d20557f5083e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b463e71faa6b4c889b8cbbaecdac42c1'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:26.670265 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c3e1505a-3477-48d5-8287-d20557f5083e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b463e71faa6b4c889b8cbbaecdac42c1'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:26.683429 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c3e1505a-3477-48d5-8287-d20557f5083e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b463e71faa6b4c889b8cbbaecdac42c1'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:26.767151 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c3e1505a-3477-48d5-8287-d20557f5083e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b463e71faa6b4c889b8cbbaecdac42c1'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:26.775306 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c3e1505a-3477-48d5-8287-d20557f5083e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b463e71faa6b4c889b8cbbaecdac42c1'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:26.781957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1060/2104] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:32:26 2022] DELETE /identity/v3/users/b463e71faa6b4c889b8cbbaecdac42c1 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:32:27.055334 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b6280-ef2b-484b-81df-3752b2b99e17 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:27.056853 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-b36b6280-ef2b-484b-81df-3752b2b99e17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:27.064912 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:27.071504 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b6280-ef2b-484b-81df-3752b2b99e17 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:27.138442 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c450cd5d-7c24-4fdb-9453-5339c166729c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:27.139014 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c450cd5d-7c24-4fdb-9453-5339c166729c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:27.139303 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c450cd5d-7c24-4fdb-9453-5339c166729c admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:27.139584 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c450cd5d-7c24-4fdb-9453-5339c166729c admin admin] PATH_INFO: `/v3/projects/34e7683fe6a04987b177adec4a4f435a` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:27.140831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c450cd5d-7c24-4fdb-9453-5339c166729c admin admin] RBAC: Authorizing `identity:delete_project(project_id=34e7683fe6a04987b177adec4a4f435a)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:27.142732 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c450cd5d-7c24-4fdb-9453-5339c166729c admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:27.231896 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e1505a-3477-48d5-8287-d20557f5083e admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:27.235007 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-c3e1505a-3477-48d5-8287-d20557f5083e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:27.243393 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-c450cd5d-7c24-4fdb-9453-5339c166729c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34e7683fe6a04987b177adec4a4f435a', 'request_id': 'req-c450cd5d-7c24-4fdb-9453-5339c166729c'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:27.249319 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e1505a-3477-48d5-8287-d20557f5083e admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:27.253567 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-c450cd5d-7c24-4fdb-9453-5339c166729c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34e7683fe6a04987b177adec4a4f435a', 'request_id': 'req-c450cd5d-7c24-4fdb-9453-5339c166729c'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:27.309056 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78406929-f159-42c4-818a-f1b9ff29d49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:27.309705 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78406929-f159-42c4-818a-f1b9ff29d49b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:27.309995 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78406929-f159-42c4-818a-f1b9ff29d49b admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:27.310295 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78406929-f159-42c4-818a-f1b9ff29d49b admin admin] PATH_INFO: `/v3/projects/90ecfd292fc9456cacd8db92f8fc7a97` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:27.311449 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78406929-f159-42c4-818a-f1b9ff29d49b admin admin] RBAC: Authorizing `identity:delete_project(project_id=90ecfd292fc9456cacd8db92f8fc7a97)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:27.313426 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78406929-f159-42c4-818a-f1b9ff29d49b admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:27.327923 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-c450cd5d-7c24-4fdb-9453-5339c166729c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34e7683fe6a04987b177adec4a4f435a', 'request_id': 'req-c450cd5d-7c24-4fdb-9453-5339c166729c'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:27.339850 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-c450cd5d-7c24-4fdb-9453-5339c166729c admin admin] The token cache is being invalidate because project 34e7683fe6a04987b177adec4a4f435a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:32:27.340228 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-c450cd5d-7c24-4fdb-9453-5339c166729c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:27.340868 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-c450cd5d-7c24-4fdb-9453-5339c166729c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:27.342390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1045/2105] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:32:27 2022] DELETE /identity/v3/projects/34e7683fe6a04987b177adec4a4f435a => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:27.347842 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810da8c-f265-44a1-818e-72f26311eb24 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:27.348168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810da8c-f265-44a1-818e-72f26311eb24 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:27.348538 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810da8c-f265-44a1-818e-72f26311eb24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:27.368146 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-1810da8c-f265-44a1-818e-72f26311eb24 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:27.369385 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-1810da8c-f265-44a1-818e-72f26311eb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:27.378297 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1046/2106] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:32:27.385957 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d1188-f951-400a-9f21-1b29a3d3a6e4 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:27.386251 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d1188-f951-400a-9f21-1b29a3d3a6e4 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:27.386585 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d1188-f951-400a-9f21-1b29a3d3a6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:27.406926 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-d23d1188-f951-400a-9f21-1b29a3d3a6e4 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:27.408214 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-d23d1188-f951-400a-9f21-1b29a3d3a6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:27.416274 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1047/2107] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:32:27.420284 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d1188-f951-400a-9f21-1b29a3d3a6e4 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:27.429209 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d1188-f951-400a-9f21-1b29a3d3a6e4 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:27.433855 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ac745-e78c-4c24-9a80-8cab01918e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:27.434483 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ac745-e78c-4c24-9a80-8cab01918e84 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:27.434769 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ac745-e78c-4c24-9a80-8cab01918e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:27.435113 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ac745-e78c-4c24-9a80-8cab01918e84 admin admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:27.435806 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0ac745-e78c-4c24-9a80-8cab01918e84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:32:27.436159 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0ac745-e78c-4c24-9a80-8cab01918e84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:27.437739 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0ac745-e78c-4c24-9a80-8cab01918e84 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:27.452169 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1048/2108] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:32:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:32:27.456695 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ac745-e78c-4c24-9a80-8cab01918e84 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:27.467026 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ac745-e78c-4c24-9a80-8cab01918e84 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:27.470445 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-78406929-f159-42c4-818a-f1b9ff29d49b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90ecfd292fc9456cacd8db92f8fc7a97', 'request_id': 'req-78406929-f159-42c4-818a-f1b9ff29d49b'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:27.476811 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-78406929-f159-42c4-818a-f1b9ff29d49b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90ecfd292fc9456cacd8db92f8fc7a97', 'request_id': 'req-78406929-f159-42c4-818a-f1b9ff29d49b'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:27.482679 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-78406929-f159-42c4-818a-f1b9ff29d49b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90ecfd292fc9456cacd8db92f8fc7a97', 'request_id': 'req-78406929-f159-42c4-818a-f1b9ff29d49b'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:27.485294 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:27.526717 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f83cd5-135d-4811-a946-bfc77f2d4d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:27.527336 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f83cd5-135d-4811-a946-bfc77f2d4d34 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:27.527621 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f83cd5-135d-4811-a946-bfc77f2d4d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:27.527913 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f83cd5-135d-4811-a946-bfc77f2d4d34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:27.528604 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f83cd5-135d-4811-a946-bfc77f2d4d34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:27.531147 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f83cd5-135d-4811-a946-bfc77f2d4d34 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:27.543884 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:27.572544 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-78406929-f159-42c4-818a-f1b9ff29d49b admin admin] The token cache is being invalidate because project 90ecfd292fc9456cacd8db92f8fc7a97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:32:27.572850 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-78406929-f159-42c4-818a-f1b9ff29d49b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:27.573479 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-78406929-f159-42c4-818a-f1b9ff29d49b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:27.575000 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1061/2109] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:32:27 2022] DELETE /identity/v3/projects/90ecfd292fc9456cacd8db92f8fc7a97 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:27.580360 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c957b-d21c-439f-8451-404d2845e502 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:27.580657 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c957b-d21c-439f-8451-404d2845e502 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:27.580948 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c957b-d21c-439f-8451-404d2845e502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:27.602590 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-3e3c957b-d21c-439f-8451-404d2845e502 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:27.603835 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-3e3c957b-d21c-439f-8451-404d2845e502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:27.613413 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1062/2110] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:32:27.623696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da92f1-9ce7-441b-9f5a-7293b15234fd None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:27.623995 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da92f1-9ce7-441b-9f5a-7293b15234fd None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:27.624882 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da92f1-9ce7-441b-9f5a-7293b15234fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:27.645011 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1049/2111] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:32:27 2022] POST /identity/v3/projects => generated 359 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:32:27.646068 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-a9da92f1-9ce7-441b-9f5a-7293b15234fd None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:27.649357 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-a9da92f1-9ce7-441b-9f5a-7293b15234fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:27.659748 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f83cd5-135d-4811-a946-bfc77f2d4d34 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:27.661706 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-41f83cd5-135d-4811-a946-bfc77f2d4d34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:27.716347 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1063/2112] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:32:27.720537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da92f1-9ce7-441b-9f5a-7293b15234fd None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:27.730655 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da92f1-9ce7-441b-9f5a-7293b15234fd None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:27.733781 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f83cd5-135d-4811-a946-bfc77f2d4d34 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:27.738778 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41176770-7c03-4540-90f2-2e42a798cc3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:27.739487 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41176770-7c03-4540-90f2-2e42a798cc3e admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:27.739673 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ac2d3-4194-42e0-a432-a76a31c5bf44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:27.740218 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41176770-7c03-4540-90f2-2e42a798cc3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:27.740567 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41176770-7c03-4540-90f2-2e42a798cc3e admin admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:27.740729 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ac2d3-4194-42e0-a432-a76a31c5bf44 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:27.741022 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ac2d3-4194-42e0-a432-a76a31c5bf44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:27.741310 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ac2d3-4194-42e0-a432-a76a31c5bf44 admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:27.741723 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41176770-7c03-4540-90f2-2e42a798cc3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:32:27.741967 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ac2d3-4194-42e0-a432-a76a31c5bf44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:27.742190 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41176770-7c03-4540-90f2-2e42a798cc3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:27.744013 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ac2d3-4194-42e0-a432-a76a31c5bf44 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:27.744266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41176770-7c03-4540-90f2-2e42a798cc3e admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:27.764320 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1064/2113] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:32:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:32:27.768993 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41176770-7c03-4540-90f2-2e42a798cc3e admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:27.779061 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41176770-7c03-4540-90f2-2e42a798cc3e admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:27.783883 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50953ab7-5cfc-4fbb-b024-a80f6a214b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:27.784500 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50953ab7-5cfc-4fbb-b024-a80f6a214b08 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:27.784785 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50953ab7-5cfc-4fbb-b024-a80f6a214b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:27.785061 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50953ab7-5cfc-4fbb-b024-a80f6a214b08 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:27.785768 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50953ab7-5cfc-4fbb-b024-a80f6a214b08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:27.787672 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50953ab7-5cfc-4fbb-b024-a80f6a214b08 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:27.824872 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1050/2114] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:32:27 2022] POST /identity/v3/users => generated 280 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:32:27.829700 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ac2d3-4194-42e0-a432-a76a31c5bf44 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:27.839916 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ac2d3-4194-42e0-a432-a76a31c5bf44 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:27.844634 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d406962f-b26a-4b2e-8d46-fd6c5243c533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:27.845216 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d406962f-b26a-4b2e-8d46-fd6c5243c533 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:27.845508 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d406962f-b26a-4b2e-8d46-fd6c5243c533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:27.845815 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d406962f-b26a-4b2e-8d46-fd6c5243c533 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:27.846566 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d406962f-b26a-4b2e-8d46-fd6c5243c533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:27.848173 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d406962f-b26a-4b2e-8d46-fd6c5243c533 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:27.858058 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1051/2115] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:32:27.861722 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d406962f-b26a-4b2e-8d46-fd6c5243c533 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:27.871473 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d406962f-b26a-4b2e-8d46-fd6c5243c533 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:27.872712 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1065/2116] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:32:27 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:32:27.877152 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50953ab7-5cfc-4fbb-b024-a80f6a214b08 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:27.932839 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad849ac7-d3a6-401d-aa30-aa7ced832fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:27.933374 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad849ac7-d3a6-401d-aa30-aa7ced832fc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:27.933657 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad849ac7-d3a6-401d-aa30-aa7ced832fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:27.933935 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad849ac7-d3a6-401d-aa30-aa7ced832fc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:27.934661 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad849ac7-d3a6-401d-aa30-aa7ced832fc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:27.936273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad849ac7-d3a6-401d-aa30-aa7ced832fc4 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:27.941279 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50953ab7-5cfc-4fbb-b024-a80f6a214b08 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:27.945951 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397238e-34b5-486b-9c3b-144ee51fb074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:27.946562 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397238e-34b5-486b-9c3b-144ee51fb074 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:27.946846 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397238e-34b5-486b-9c3b-144ee51fb074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:27.947132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397238e-34b5-486b-9c3b-144ee51fb074 admin admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:27.947253 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1052/2117] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:27 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 15:32:27.947792 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c397238e-34b5-486b-9c3b-144ee51fb074 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:27.949835 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c397238e-34b5-486b-9c3b-144ee51fb074 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:27.951896 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad849ac7-d3a6-401d-aa30-aa7ced832fc4 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:27.961846 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad849ac7-d3a6-401d-aa30-aa7ced832fc4 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:27.966497 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6358cb5-ea0f-4cbd-9b99-be790888a442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:27.967231 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6358cb5-ea0f-4cbd-9b99-be790888a442 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:27.967512 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6358cb5-ea0f-4cbd-9b99-be790888a442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:27.967788 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6358cb5-ea0f-4cbd-9b99-be790888a442 admin admin] PATH_INFO: `/v3/projects/c78a5abb39b94d3aa1f324896b279a9f/users/11883954b8ce448b9c6b4e3d67bec0c7/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:27.982352 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6358cb5-ea0f-4cbd-9b99-be790888a442 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c78a5abb39b94d3aa1f324896b279a9f, user_id=11883954b8ce448b9c6b4e3d67bec0c7, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:27.985549 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6358cb5-ea0f-4cbd-9b99-be790888a442 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:28.024777 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1066/2118] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:32:27 2022] POST /identity/v3/users => generated 292 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:32:28.029072 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397238e-34b5-486b-9c3b-144ee51fb074 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:28.038741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397238e-34b5-486b-9c3b-144ee51fb074 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:28.043572 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a69ab-08eb-434b-857e-e3925affc1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:28.044193 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a69ab-08eb-434b-857e-e3925affc1f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:28.044478 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a69ab-08eb-434b-857e-e3925affc1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:28.044759 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a69ab-08eb-434b-857e-e3925affc1f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:28.045534 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53a69ab-08eb-434b-857e-e3925affc1f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:28.047715 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53a69ab-08eb-434b-857e-e3925affc1f5 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:28.048249 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1053/2119] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:32:27 2022] PUT /identity/v3/projects/c78a5abb39b94d3aa1f324896b279a9f/users/11883954b8ce448b9c6b4e3d67bec0c7/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:32:28.063227 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1067/2120] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 15:32:28.063405 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d373cf2-ae4b-4142-abc2-6806277c78b4 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:28.063405 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d373cf2-ae4b-4142-abc2-6806277c78b4 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:28.063405 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d373cf2-ae4b-4142-abc2-6806277c78b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:28.065148 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a69ab-08eb-434b-857e-e3925affc1f5 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:28.075819 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a69ab-08eb-434b-857e-e3925affc1f5 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:28.093748 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-5d373cf2-ae4b-4142-abc2-6806277c78b4 None None] MFA Rules not processed for user `11883954b8ce448b9c6b4e3d67bec0c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:28.094970 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-5d373cf2-ae4b-4142-abc2-6806277c78b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:28.106947 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:INSERT=5,keystone:DELETE=1,keystone:SELECT=94 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:28.126597 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:28.138384 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d239a0d-179a-4aee-bc27-078c8cb549ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:28.138973 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d239a0d-179a-4aee-bc27-078c8cb549ec admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:28.139258 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d239a0d-179a-4aee-bc27-078c8cb549ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:28.139539 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d239a0d-179a-4aee-bc27-078c8cb549ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:28.140778 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d239a0d-179a-4aee-bc27-078c8cb549ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:28.142416 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d239a0d-179a-4aee-bc27-078c8cb549ec admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:28.152459 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1068/2121] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:28 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 15:32:28.157082 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d239a0d-179a-4aee-bc27-078c8cb549ec admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:28.162840 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1054/2122] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:28 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:28.170623 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d239a0d-179a-4aee-bc27-078c8cb549ec admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:28.178635 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649dcdcf-1417-4541-b769-ba19768beaf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:28.178635 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649dcdcf-1417-4541-b769-ba19768beaf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:28.178635 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649dcdcf-1417-4541-b769-ba19768beaf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:28.178635 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649dcdcf-1417-4541-b769-ba19768beaf7 admin admin] PATH_INFO: `/v3/projects/6d0c97d5fc004afe98493db0c2bfafa8/users/58d96ed807b74a25b1b79ce75cbfd0db/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:28.197106 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649dcdcf-1417-4541-b769-ba19768beaf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d0c97d5fc004afe98493db0c2bfafa8, user_id=58d96ed807b74a25b1b79ce75cbfd0db, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:28.199872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649dcdcf-1417-4541-b769-ba19768beaf7 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:28.203052 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d373cf2-ae4b-4142-abc2-6806277c78b4 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:28.204573 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-5d373cf2-ae4b-4142-abc2-6806277c78b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:28.260997 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d373cf2-ae4b-4142-abc2-6806277c78b4 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:28.265205 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b480c-2cf9-46d6-8c01-0bbce905b3a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:28.265815 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b480c-2cf9-46d6-8c01-0bbce905b3a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:28.266098 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b480c-2cf9-46d6-8c01-0bbce905b3a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:28.266370 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b480c-2cf9-46d6-8c01-0bbce905b3a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:28.274772 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36b480c-2cf9-46d6-8c01-0bbce905b3a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:28.276220 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36b480c-2cf9-46d6-8c01-0bbce905b3a0 service cinder] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:28.285833 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1055/2123] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:32:28 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 15:32:28.289656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1069/2124] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:32:28 2022] PUT /identity/v3/projects/6d0c97d5fc004afe98493db0c2bfafa8/users/58d96ed807b74a25b1b79ce75cbfd0db/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:32:28.299959 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5d65a-7edc-4820-bcd3-3d4b258a41d0 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:28.300367 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5d65a-7edc-4820-bcd3-3d4b258a41d0 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:28.300696 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5d65a-7edc-4820-bcd3-3d4b258a41d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:28.302610 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649dcdcf-1417-4541-b769-ba19768beaf7 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:28.333424 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-80b5d65a-7edc-4820-bcd3-3d4b258a41d0 None None] MFA Rules not processed for user `58d96ed807b74a25b1b79ce75cbfd0db`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:28.334656 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-80b5d65a-7edc-4820-bcd3-3d4b258a41d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:28.364385 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649dcdcf-1417-4541-b769-ba19768beaf7 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:28.369327 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8021efb-3efd-4aa4-85c2-2961c1df3f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:28.369966 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8021efb-3efd-4aa4-85c2-2961c1df3f57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:28.370251 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8021efb-3efd-4aa4-85c2-2961c1df3f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:28.370530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8021efb-3efd-4aa4-85c2-2961c1df3f57 admin admin] PATH_INFO: `/v3/users/11883954b8ce448b9c6b4e3d67bec0c7` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:28.371727 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8021efb-3efd-4aa4-85c2-2961c1df3f57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11883954b8ce448b9c6b4e3d67bec0c7)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:28.373689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8021efb-3efd-4aa4-85c2-2961c1df3f57 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:28.401386 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1056/2125] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:32:28.405892 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b5d65a-7edc-4820-bcd3-3d4b258a41d0 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:28.416090 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b5d65a-7edc-4820-bcd3-3d4b258a41d0 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:28.420700 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5ba2c-dc86-4631-9dcc-4527ebea9b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:28.421302 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5ba2c-dc86-4631-9dcc-4527ebea9b27 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:28.421618 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5ba2c-dc86-4631-9dcc-4527ebea9b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:28.421946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5ba2c-dc86-4631-9dcc-4527ebea9b27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:28.422654 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d5ba2c-dc86-4631-9dcc-4527ebea9b27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:28.424640 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d5ba2c-dc86-4631-9dcc-4527ebea9b27 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:28.506040 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1057/2126] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:32:28 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:32:28.509895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5ba2c-dc86-4631-9dcc-4527ebea9b27 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:28.554479 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:28.571196 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5ba2c-dc86-4631-9dcc-4527ebea9b27 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:28.575834 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7088d2ab-a237-447b-8ca6-31cac0c4356d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:28.576467 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088d2ab-a237-447b-8ca6-31cac0c4356d admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:28.576763 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088d2ab-a237-447b-8ca6-31cac0c4356d admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:28.577035 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088d2ab-a237-447b-8ca6-31cac0c4356d admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:28.578415 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7088d2ab-a237-447b-8ca6-31cac0c4356d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:28.580350 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7088d2ab-a237-447b-8ca6-31cac0c4356d admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:28.597704 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-d8021efb-3efd-4aa4-85c2-2961c1df3f57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11883954b8ce448b9c6b4e3d67bec0c7'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:28.598362 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-d8021efb-3efd-4aa4-85c2-2961c1df3f57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11883954b8ce448b9c6b4e3d67bec0c7'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:28.610330 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-d8021efb-3efd-4aa4-85c2-2961c1df3f57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11883954b8ce448b9c6b4e3d67bec0c7'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:28.635704 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1058/2127] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:32:28 2022] POST /identity/v3/users => generated 299 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:32:28.639402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7088d2ab-a237-447b-8ca6-31cac0c4356d admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:28.641603 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-7088d2ab-a237-447b-8ca6-31cac0c4356d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:28.653400 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7088d2ab-a237-447b-8ca6-31cac0c4356d admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:28.658080 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ecf412-b567-4ab3-8f9a-bc97c6361e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:28.658658 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ecf412-b567-4ab3-8f9a-bc97c6361e89 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:28.660292 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ecf412-b567-4ab3-8f9a-bc97c6361e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:28.660292 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ecf412-b567-4ab3-8f9a-bc97c6361e89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:28.660292 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ecf412-b567-4ab3-8f9a-bc97c6361e89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:28.661543 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ecf412-b567-4ab3-8f9a-bc97c6361e89 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:28.676626 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1059/2128] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 15:32:28.677852 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ecf412-b567-4ab3-8f9a-bc97c6361e89 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:28.679175 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-d8021efb-3efd-4aa4-85c2-2961c1df3f57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11883954b8ce448b9c6b4e3d67bec0c7'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:28.688972 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-d8021efb-3efd-4aa4-85c2-2961c1df3f57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11883954b8ce448b9c6b4e3d67bec0c7'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:28.690656 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ecf412-b567-4ab3-8f9a-bc97c6361e89 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:28.695751 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c6176-26ca-4005-853d-9cbc12fde3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:28.696127 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1070/2129] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:32:28 2022] DELETE /identity/v3/users/11883954b8ce448b9c6b4e3d67bec0c7 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:32:28.699294 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c6176-26ca-4005-853d-9cbc12fde3e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:28.699583 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c6176-26ca-4005-853d-9cbc12fde3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:28.699866 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c6176-26ca-4005-853d-9cbc12fde3e3 admin admin] PATH_INFO: `/v3/projects/840683ef03c0495e8a677774f4bce4f6/users/1a6af170e68d4447be87dfa4307761be/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:28.708662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45832928-2c7e-4ca7-9b88-d72ff8a0a606 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:28.709068 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45832928-2c7e-4ca7-9b88-d72ff8a0a606 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:28.709422 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45832928-2c7e-4ca7-9b88-d72ff8a0a606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:28.715472 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8c6176-26ca-4005-853d-9cbc12fde3e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=840683ef03c0495e8a677774f4bce4f6, user_id=1a6af170e68d4447be87dfa4307761be, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:28.718530 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8c6176-26ca-4005-853d-9cbc12fde3e3 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:28.729509 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-45832928-2c7e-4ca7-9b88-d72ff8a0a606 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:28.730703 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-45832928-2c7e-4ca7-9b88-d72ff8a0a606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:28.744083 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:28.792110 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1071/2130] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:28.802907 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1060/2131] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:32:28 2022] PUT /identity/v3/projects/840683ef03c0495e8a677774f4bce4f6/users/1a6af170e68d4447be87dfa4307761be/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:32:28.803846 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45832928-2c7e-4ca7-9b88-d72ff8a0a606 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:28.805200 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-45832928-2c7e-4ca7-9b88-d72ff8a0a606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:28.808384 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c6176-26ca-4005-853d-9cbc12fde3e3 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:28.873967 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45832928-2c7e-4ca7-9b88-d72ff8a0a606 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:28.880872 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c6176-26ca-4005-853d-9cbc12fde3e3 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:28.887498 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59111090-3a2b-434a-90e3-54a270585572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:28.888114 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59111090-3a2b-434a-90e3-54a270585572 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:28.888409 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59111090-3a2b-434a-90e3-54a270585572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:28.888690 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59111090-3a2b-434a-90e3-54a270585572 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:28.889397 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59111090-3a2b-434a-90e3-54a270585572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:28.891246 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59111090-3a2b-434a-90e3-54a270585572 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:28.902894 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1061/2132] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:28 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 15:32:28.907132 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59111090-3a2b-434a-90e3-54a270585572 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:28.918113 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59111090-3a2b-434a-90e3-54a270585572 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:28.924413 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c4543a-ab74-4e58-b8f2-2c4fd45d81f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:28.925252 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4543a-ab74-4e58-b8f2-2c4fd45d81f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:28.946811 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4543a-ab74-4e58-b8f2-2c4fd45d81f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:28.946811 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4543a-ab74-4e58-b8f2-2c4fd45d81f7 admin admin] PATH_INFO: `/v3/projects/840683ef03c0495e8a677774f4bce4f6/users/1a6af170e68d4447be87dfa4307761be/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:28.946811 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c4543a-ab74-4e58-b8f2-2c4fd45d81f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=840683ef03c0495e8a677774f4bce4f6, user_id=1a6af170e68d4447be87dfa4307761be, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:28.946811 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c4543a-ab74-4e58-b8f2-2c4fd45d81f7 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:28.947264 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563abc71-66b4-4929-8ed2-2c9317121303 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:28.947264 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563abc71-66b4-4929-8ed2-2c9317121303 service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:28.947264 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563abc71-66b4-4929-8ed2-2c9317121303 service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:28.947264 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563abc71-66b4-4929-8ed2-2c9317121303 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:28.951092 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1062/2133] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:32:28 2022] PUT /identity/v3/projects/840683ef03c0495e8a677774f4bce4f6/users/1a6af170e68d4447be87dfa4307761be/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:28.951232 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563abc71-66b4-4929-8ed2-2c9317121303 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:28.951232 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563abc71-66b4-4929-8ed2-2c9317121303 service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:28.956207 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd7672-ef05-4318-b44d-29b468bab5ed None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:28.956524 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd7672-ef05-4318-b44d-29b468bab5ed None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:28.956865 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd7672-ef05-4318-b44d-29b468bab5ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:28.960889 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1072/2134] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:32:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 15:32:28.990502 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-89dd7672-ef05-4318-b44d-29b468bab5ed None None] MFA Rules not processed for user `1a6af170e68d4447be87dfa4307761be`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:28.991727 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-89dd7672-ef05-4318-b44d-29b468bab5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:29.066421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1063/2135] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:28 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:32:29.077403 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563abc71-66b4-4929-8ed2-2c9317121303 service neutron] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:29.078818 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-563abc71-66b4-4929-8ed2-2c9317121303 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:29.138248 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563abc71-66b4-4929-8ed2-2c9317121303 service neutron] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:29.175705 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:29.195709 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3748e78f-d357-4e39-afa3-084ee296e826 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:29.195709 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748e78f-d357-4e39-afa3-084ee296e826 service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:29.195709 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748e78f-d357-4e39-afa3-084ee296e826 service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:29.195709 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748e78f-d357-4e39-afa3-084ee296e826 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:29.203906 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3748e78f-d357-4e39-afa3-084ee296e826 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:29.205417 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3748e78f-d357-4e39-afa3-084ee296e826 service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:29.216118 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1073/2136] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:32:29 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:32:29.246138 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd7672-ef05-4318-b44d-29b468bab5ed None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:29.251976 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:29.309129 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd7672-ef05-4318-b44d-29b468bab5ed None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:29.313751 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f8a971-aab5-4a37-8575-d407d011a23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:29.314408 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8a971-aab5-4a37-8575-d407d011a23a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:29.314919 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8a971-aab5-4a37-8575-d407d011a23a admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:29.315370 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8a971-aab5-4a37-8575-d407d011a23a admin admin] PATH_INFO: `/v3/users/58d96ed807b74a25b1b79ce75cbfd0db` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:29.317056 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f8a971-aab5-4a37-8575-d407d011a23a admin admin] RBAC: Authorizing `identity:delete_user(user_id=58d96ed807b74a25b1b79ce75cbfd0db)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:29.318998 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f8a971-aab5-4a37-8575-d407d011a23a admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:29.432214 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3748e78f-d357-4e39-afa3-084ee296e826 service cinder] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:29.433700 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-3748e78f-d357-4e39-afa3-084ee296e826 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:29.447365 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3748e78f-d357-4e39-afa3-084ee296e826 service cinder] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:29.451876 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a197a-8a9e-4d40-86c4-4183694d39fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:29.453054 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a197a-8a9e-4d40-86c4-4183694d39fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:29.453482 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a197a-8a9e-4d40-86c4-4183694d39fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:29.453762 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a197a-8a9e-4d40-86c4-4183694d39fd admin admin] PATH_INFO: `/v3/projects/c78a5abb39b94d3aa1f324896b279a9f` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:29.454948 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840a197a-8a9e-4d40-86c4-4183694d39fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c78a5abb39b94d3aa1f324896b279a9f)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:29.456881 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840a197a-8a9e-4d40-86c4-4183694d39fd admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:29.518143 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-96f8a971-aab5-4a37-8575-d407d011a23a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58d96ed807b74a25b1b79ce75cbfd0db'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:29.525054 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-96f8a971-aab5-4a37-8575-d407d011a23a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58d96ed807b74a25b1b79ce75cbfd0db'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:29.525737 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-96f8a971-aab5-4a37-8575-d407d011a23a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58d96ed807b74a25b1b79ce75cbfd0db'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:29.538844 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-96f8a971-aab5-4a37-8575-d407d011a23a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58d96ed807b74a25b1b79ce75cbfd0db'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:29.539498 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-96f8a971-aab5-4a37-8575-d407d011a23a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58d96ed807b74a25b1b79ce75cbfd0db'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:29.545688 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-840a197a-8a9e-4d40-86c4-4183694d39fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c78a5abb39b94d3aa1f324896b279a9f', 'request_id': 'req-840a197a-8a9e-4d40-86c4-4183694d39fd'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:29.551871 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-840a197a-8a9e-4d40-86c4-4183694d39fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c78a5abb39b94d3aa1f324896b279a9f', 'request_id': 'req-840a197a-8a9e-4d40-86c4-4183694d39fd'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:29.557591 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-840a197a-8a9e-4d40-86c4-4183694d39fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c78a5abb39b94d3aa1f324896b279a9f', 'request_id': 'req-840a197a-8a9e-4d40-86c4-4183694d39fd'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:29.602033 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1064/2137] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:32:29 2022] DELETE /identity/v3/users/58d96ed807b74a25b1b79ce75cbfd0db => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:29.606211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccffcff-57bc-4abb-a260-5761465a30eb None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:29.606566 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccffcff-57bc-4abb-a260-5761465a30eb None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:29.606964 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccffcff-57bc-4abb-a260-5761465a30eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:29.626983 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-7ccffcff-57bc-4abb-a260-5761465a30eb None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:29.630468 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-7ccffcff-57bc-4abb-a260-5761465a30eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:29.652589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-840a197a-8a9e-4d40-86c4-4183694d39fd admin admin] The token cache is being invalidate because project c78a5abb39b94d3aa1f324896b279a9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:32:29.652926 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-840a197a-8a9e-4d40-86c4-4183694d39fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:29.653561 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-840a197a-8a9e-4d40-86c4-4183694d39fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:29.655130 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1074/2138] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:32:29 2022] DELETE /identity/v3/projects/c78a5abb39b94d3aa1f324896b279a9f => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:32:29.661778 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0171d5-10fd-4aee-94da-8106192fdc43 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:29.662144 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0171d5-10fd-4aee-94da-8106192fdc43 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:29.662439 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0171d5-10fd-4aee-94da-8106192fdc43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:29.682410 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-5a0171d5-10fd-4aee-94da-8106192fdc43 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:29.683628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-5a0171d5-10fd-4aee-94da-8106192fdc43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:29.689567 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1065/2139] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:32:29.692814 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1075/2140] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:32:29.701093 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d903f6-cec8-4043-bddd-61446d7abf74 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:29.701390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d903f6-cec8-4043-bddd-61446d7abf74 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:29.701676 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d903f6-cec8-4043-bddd-61446d7abf74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:29.702614 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0171d5-10fd-4aee-94da-8106192fdc43 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:29.703952 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-5a0171d5-10fd-4aee-94da-8106192fdc43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:29.722670 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-f5d903f6-cec8-4043-bddd-61446d7abf74 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:29.723892 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-f5d903f6-cec8-4043-bddd-61446d7abf74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:29.734829 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1066/2141] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:29.738859 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d903f6-cec8-4043-bddd-61446d7abf74 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:29.750801 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d903f6-cec8-4043-bddd-61446d7abf74 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:29.755790 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fae65f8-928e-4552-b528-7f240657d31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:29.756441 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fae65f8-928e-4552-b528-7f240657d31c admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:29.756725 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fae65f8-928e-4552-b528-7f240657d31c admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:29.757013 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fae65f8-928e-4552-b528-7f240657d31c admin admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:29.757715 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fae65f8-928e-4552-b528-7f240657d31c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:32:29.758026 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fae65f8-928e-4552-b528-7f240657d31c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:29.759619 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fae65f8-928e-4552-b528-7f240657d31c admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:29.771189 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0171d5-10fd-4aee-94da-8106192fdc43 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:29.775189 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b17cd-7114-4a0e-acad-217af8134bca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:29.775903 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b17cd-7114-4a0e-acad-217af8134bca service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:29.776288 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b17cd-7114-4a0e-acad-217af8134bca service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:29.776639 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b17cd-7114-4a0e-acad-217af8134bca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:29.783632 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1067/2142] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:32:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:32:29.787969 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fae65f8-928e-4552-b528-7f240657d31c admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:29.789376 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b17cd-7114-4a0e-acad-217af8134bca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:29.791057 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b17cd-7114-4a0e-acad-217af8134bca service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:29.794441 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:29.798321 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fae65f8-928e-4552-b528-7f240657d31c admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:29.802874 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1076/2143] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:32:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:32:29.807925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e41c32-067b-4f0f-83b9-44c6f18cf966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:29.807925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e41c32-067b-4f0f-83b9-44c6f18cf966 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:29.807925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e41c32-067b-4f0f-83b9-44c6f18cf966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:29.807925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e41c32-067b-4f0f-83b9-44c6f18cf966 admin admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:29.808353 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e41c32-067b-4f0f-83b9-44c6f18cf966 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:29.808353 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e41c32-067b-4f0f-83b9-44c6f18cf966 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:29.935108 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1068/2144] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:32:29 2022] POST /identity/v3/projects => generated 397 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:32:29.939401 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b17cd-7114-4a0e-acad-217af8134bca service neutron] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:29.949951 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b17cd-7114-4a0e-acad-217af8134bca service neutron] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:30.024568 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:30.032574 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09d8e2-6798-4744-aab4-ce083c098ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:30.033090 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09d8e2-6798-4744-aab4-ce083c098ffd admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:30.033371 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09d8e2-6798-4744-aab4-ce083c098ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:30.033667 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09d8e2-6798-4744-aab4-ce083c098ffd admin admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:30.034354 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba09d8e2-6798-4744-aab4-ce083c098ffd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:30.036416 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba09d8e2-6798-4744-aab4-ce083c098ffd admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:30.126825 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1077/2145] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:32:29 2022] POST /identity/v3/users => generated 299 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:32:30.131347 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e41c32-067b-4f0f-83b9-44c6f18cf966 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:30.142145 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e41c32-067b-4f0f-83b9-44c6f18cf966 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:30.153842 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78c415d-51b0-45d9-8d9c-efc307da3d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:30.154602 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c415d-51b0-45d9-8d9c-efc307da3d2c admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:30.154892 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c415d-51b0-45d9-8d9c-efc307da3d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:30.155690 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c415d-51b0-45d9-8d9c-efc307da3d2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:30.156439 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78c415d-51b0-45d9-8d9c-efc307da3d2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:30.158064 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78c415d-51b0-45d9-8d9c-efc307da3d2c admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:30.168763 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1069/2146] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 15:32:30.173340 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09d8e2-6798-4744-aab4-ce083c098ffd admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:30.183355 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09d8e2-6798-4744-aab4-ce083c098ffd admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:30.188250 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e65626-4860-4fc3-8d01-07d376a5467d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:30.189063 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e65626-4860-4fc3-8d01-07d376a5467d admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:30.189180 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e65626-4860-4fc3-8d01-07d376a5467d admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:30.189395 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e65626-4860-4fc3-8d01-07d376a5467d admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:30.190090 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e65626-4860-4fc3-8d01-07d376a5467d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:30.191696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e65626-4860-4fc3-8d01-07d376a5467d admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:30.201477 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1078/2147] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:32:30.206228 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78c415d-51b0-45d9-8d9c-efc307da3d2c admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:30.216241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78c415d-51b0-45d9-8d9c-efc307da3d2c admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:30.221543 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf552e9-3991-40f4-a3b9-d7a8d4bbabd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:30.222193 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf552e9-3991-40f4-a3b9-d7a8d4bbabd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:30.222665 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf552e9-3991-40f4-a3b9-d7a8d4bbabd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:30.222986 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf552e9-3991-40f4-a3b9-d7a8d4bbabd8 admin admin] PATH_INFO: `/v3/projects/c5bdf7d03cb745eba3d2a6865ba84766/users/459a32aeadda435c8b54bbb1ee7a63ef/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:30.237708 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf552e9-3991-40f4-a3b9-d7a8d4bbabd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5bdf7d03cb745eba3d2a6865ba84766, user_id=459a32aeadda435c8b54bbb1ee7a63ef, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:30.240788 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf552e9-3991-40f4-a3b9-d7a8d4bbabd8 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:30.242757 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e65626-4860-4fc3-8d01-07d376a5467d admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:30.244603 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-79e65626-4860-4fc3-8d01-07d376a5467d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:30.259623 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e65626-4860-4fc3-8d01-07d376a5467d admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:30.265499 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300d67c-2981-4575-bcbf-45a287cbd56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:30.266236 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300d67c-2981-4575-bcbf-45a287cbd56e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:30.266522 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300d67c-2981-4575-bcbf-45a287cbd56e admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:30.266799 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300d67c-2981-4575-bcbf-45a287cbd56e admin admin] PATH_INFO: `/v3/projects/6d0c97d5fc004afe98493db0c2bfafa8` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:30.267975 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300d67c-2981-4575-bcbf-45a287cbd56e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d0c97d5fc004afe98493db0c2bfafa8)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:30.271196 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300d67c-2981-4575-bcbf-45a287cbd56e admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:30.299421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1070/2148] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:32:30 2022] PUT /identity/v3/projects/c5bdf7d03cb745eba3d2a6865ba84766/users/459a32aeadda435c8b54bbb1ee7a63ef/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:32:30.309621 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b66238-bc59-4db6-a60b-3f7d2039d5b4 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:30.309999 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b66238-bc59-4db6-a60b-3f7d2039d5b4 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:30.310304 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b66238-bc59-4db6-a60b-3f7d2039d5b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:30.338472 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-c7b66238-bc59-4db6-a60b-3f7d2039d5b4 None None] MFA Rules not processed for user `459a32aeadda435c8b54bbb1ee7a63ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:30.339698 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-c7b66238-bc59-4db6-a60b-3f7d2039d5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:30.385745 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c300d67c-2981-4575-bcbf-45a287cbd56e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d0c97d5fc004afe98493db0c2bfafa8', 'request_id': 'req-c300d67c-2981-4575-bcbf-45a287cbd56e'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:30.393124 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c300d67c-2981-4575-bcbf-45a287cbd56e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d0c97d5fc004afe98493db0c2bfafa8', 'request_id': 'req-c300d67c-2981-4575-bcbf-45a287cbd56e'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:30.398890 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c300d67c-2981-4575-bcbf-45a287cbd56e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d0c97d5fc004afe98493db0c2bfafa8', 'request_id': 'req-c300d67c-2981-4575-bcbf-45a287cbd56e'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:30.405995 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1071/2149] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:30 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:32:30.417403 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b66238-bc59-4db6-a60b-3f7d2039d5b4 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:30.418741 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-c7b66238-bc59-4db6-a60b-3f7d2039d5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:30.463497 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c300d67c-2981-4575-bcbf-45a287cbd56e admin admin] The token cache is being invalidate because project 6d0c97d5fc004afe98493db0c2bfafa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:32:30.464105 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c300d67c-2981-4575-bcbf-45a287cbd56e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:30.464537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c300d67c-2981-4575-bcbf-45a287cbd56e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:30.469883 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1079/2150] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:32:30 2022] DELETE /identity/v3/projects/6d0c97d5fc004afe98493db0c2bfafa8 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:32:30.497664 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300d67c-2981-4575-bcbf-45a287cbd56e admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:30.497664 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-c300d67c-2981-4575-bcbf-45a287cbd56e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:30.497991 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b66238-bc59-4db6-a60b-3f7d2039d5b4 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:30.532851 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ed4cb2-31f2-4e3c-a473-e93319b6aaf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:30.533469 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed4cb2-31f2-4e3c-a473-e93319b6aaf6 service cinder] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:30.533752 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed4cb2-31f2-4e3c-a473-e93319b6aaf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:30.534027 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed4cb2-31f2-4e3c-a473-e93319b6aaf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:30.535808 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-23ed4cb2-31f2-4e3c-a473-e93319b6aaf6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:30.540604 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300d67c-2981-4575-bcbf-45a287cbd56e admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:30.542039 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:30.545412 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed1fd78-c29c-446b-ac46-0dd51a8d1bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:30.546055 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1fd78-c29c-446b-ac46-0dd51a8d1bba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:30.546341 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1fd78-c29c-446b-ac46-0dd51a8d1bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:30.546622 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1fd78-c29c-446b-ac46-0dd51a8d1bba admin admin] PATH_INFO: `/v3/users/1a6af170e68d4447be87dfa4307761be` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:30.547806 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed1fd78-c29c-446b-ac46-0dd51a8d1bba admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a6af170e68d4447be87dfa4307761be)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:30.549815 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed1fd78-c29c-446b-ac46-0dd51a8d1bba admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:30.594821 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ed4cb2-31f2-4e3c-a473-e93319b6aaf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:30.596292 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ed4cb2-31f2-4e3c-a473-e93319b6aaf6 service cinder] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:30.605507 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1072/2151] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:32:30 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 15:32:30.754983 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-7ed1fd78-c29c-446b-ac46-0dd51a8d1bba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a6af170e68d4447be87dfa4307761be'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:30.755823 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-7ed1fd78-c29c-446b-ac46-0dd51a8d1bba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a6af170e68d4447be87dfa4307761be'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:30.766860 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=81,keystone:INSERT=4,keystone:DELETE=15 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:30.768926 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-7ed1fd78-c29c-446b-ac46-0dd51a8d1bba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a6af170e68d4447be87dfa4307761be'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:30.881668 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-7ed1fd78-c29c-446b-ac46-0dd51a8d1bba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a6af170e68d4447be87dfa4307761be'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:30.889076 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-7ed1fd78-c29c-446b-ac46-0dd51a8d1bba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a6af170e68d4447be87dfa4307761be'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:30.896386 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1080/2152] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:32:30 2022] DELETE /identity/v3/users/1a6af170e68d4447be87dfa4307761be => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:32:31.351533 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ed4cb2-31f2-4e3c-a473-e93319b6aaf6 service cinder] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:31.366932 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ed4cb2-31f2-4e3c-a473-e93319b6aaf6 service cinder] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:31.429278 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70de7d3f-b232-4687-afec-0b1cad65757f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:31.433080 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de7d3f-b232-4687-afec-0b1cad65757f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:31.433080 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de7d3f-b232-4687-afec-0b1cad65757f admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:31.433080 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de7d3f-b232-4687-afec-0b1cad65757f admin admin] PATH_INFO: `/v3/projects/840683ef03c0495e8a677774f4bce4f6` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:31.433080 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70de7d3f-b232-4687-afec-0b1cad65757f admin admin] RBAC: Authorizing `identity:delete_project(project_id=840683ef03c0495e8a677774f4bce4f6)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:31.433632 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70de7d3f-b232-4687-afec-0b1cad65757f admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:31.568393 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-70de7d3f-b232-4687-afec-0b1cad65757f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '840683ef03c0495e8a677774f4bce4f6', 'request_id': 'req-70de7d3f-b232-4687-afec-0b1cad65757f'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:31.574891 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-70de7d3f-b232-4687-afec-0b1cad65757f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '840683ef03c0495e8a677774f4bce4f6', 'request_id': 'req-70de7d3f-b232-4687-afec-0b1cad65757f'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:31.663880 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-70de7d3f-b232-4687-afec-0b1cad65757f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '840683ef03c0495e8a677774f4bce4f6', 'request_id': 'req-70de7d3f-b232-4687-afec-0b1cad65757f'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:31.675490 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-70de7d3f-b232-4687-afec-0b1cad65757f admin admin] The token cache is being invalidate because project 840683ef03c0495e8a677774f4bce4f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:32:31.675781 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-70de7d3f-b232-4687-afec-0b1cad65757f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:31.676443 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-70de7d3f-b232-4687-afec-0b1cad65757f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:31.677968 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1073/2153] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:32:31 2022] DELETE /identity/v3/projects/840683ef03c0495e8a677774f4bce4f6 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:31.682780 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6cdcb-2d34-41d7-91dd-aeecd5e7d8e9 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:31.683278 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6cdcb-2d34-41d7-91dd-aeecd5e7d8e9 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:31.683382 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6cdcb-2d34-41d7-91dd-aeecd5e7d8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:31.706050 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-43d6cdcb-2d34-41d7-91dd-aeecd5e7d8e9 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:31.707272 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-43d6cdcb-2d34-41d7-91dd-aeecd5e7d8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:31.766651 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1081/2154] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:31.774309 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29754041-44b0-47e6-a7ca-5c7d747b732b None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:31.775552 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29754041-44b0-47e6-a7ca-5c7d747b732b None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:31.775552 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29754041-44b0-47e6-a7ca-5c7d747b732b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:31.796401 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-29754041-44b0-47e6-a7ca-5c7d747b732b None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:31.797631 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-29754041-44b0-47e6-a7ca-5c7d747b732b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:31.805857 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1074/2155] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:31.814215 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d6cdcb-2d34-41d7-91dd-aeecd5e7d8e9 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:31.824415 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d6cdcb-2d34-41d7-91dd-aeecd5e7d8e9 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:31.830047 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237a03e-3560-4c05-9209-769b5f0261ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:31.830314 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237a03e-3560-4c05-9209-769b5f0261ee admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:31.830430 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237a03e-3560-4c05-9209-769b5f0261ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:31.830643 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237a03e-3560-4c05-9209-769b5f0261ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:31.831406 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4237a03e-3560-4c05-9209-769b5f0261ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:32:31.831721 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4237a03e-3560-4c05-9209-769b5f0261ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:31.833533 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4237a03e-3560-4c05-9209-769b5f0261ee admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:31.851181 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1082/2156] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:32:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:32:31.852564 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29754041-44b0-47e6-a7ca-5c7d747b732b None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:31.866973 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29754041-44b0-47e6-a7ca-5c7d747b732b None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:31.872504 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7622caaa-4882-4c7c-ad80-ff89a007b74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:31.873057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7622caaa-4882-4c7c-ad80-ff89a007b74d admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:31.873341 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7622caaa-4882-4c7c-ad80-ff89a007b74d admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:31.873647 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7622caaa-4882-4c7c-ad80-ff89a007b74d admin admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:31.874590 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7622caaa-4882-4c7c-ad80-ff89a007b74d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:31.880532 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7622caaa-4882-4c7c-ad80-ff89a007b74d admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:32.002819 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1075/2157] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:32:31 2022] POST /identity/v3/projects => generated 371 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:32:32.007109 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237a03e-3560-4c05-9209-769b5f0261ee admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:32.017454 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237a03e-3560-4c05-9209-769b5f0261ee admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:32.099904 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bde859-3209-4e9b-8c0d-edc80d6dd1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:32.100561 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bde859-3209-4e9b-8c0d-edc80d6dd1d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:32.100849 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bde859-3209-4e9b-8c0d-edc80d6dd1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:32.101126 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bde859-3209-4e9b-8c0d-edc80d6dd1d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:32.101825 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bde859-3209-4e9b-8c0d-edc80d6dd1d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:32.103902 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bde859-3209-4e9b-8c0d-edc80d6dd1d9 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:32.204516 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1083/2158] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:32:32 2022] POST /identity/v3/users => generated 286 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:32:32.207291 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7622caaa-4882-4c7c-ad80-ff89a007b74d admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:32.218318 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7622caaa-4882-4c7c-ad80-ff89a007b74d admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:32.223194 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a535e5-d8dd-48c6-b22c-9ed4d45dcb6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:32.223876 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a535e5-d8dd-48c6-b22c-9ed4d45dcb6e admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:32.224262 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a535e5-d8dd-48c6-b22c-9ed4d45dcb6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:32.224611 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a535e5-d8dd-48c6-b22c-9ed4d45dcb6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:32.225406 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a535e5-d8dd-48c6-b22c-9ed4d45dcb6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:32.227106 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a535e5-d8dd-48c6-b22c-9ed4d45dcb6e admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:32.229367 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:32.241586 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1076/2159] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:32:32.242711 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bde859-3209-4e9b-8c0d-edc80d6dd1d9 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:32.253462 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bde859-3209-4e9b-8c0d-edc80d6dd1d9 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:32.258784 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144151a-7581-4a89-84c1-1e2579c11822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:32.259952 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144151a-7581-4a89-84c1-1e2579c11822 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:32.260047 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144151a-7581-4a89-84c1-1e2579c11822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:32.260511 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144151a-7581-4a89-84c1-1e2579c11822 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:32.261601 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e144151a-7581-4a89-84c1-1e2579c11822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:32.264051 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e144151a-7581-4a89-84c1-1e2579c11822 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:32.277359 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1084/2160] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:32:32.282174 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a535e5-d8dd-48c6-b22c-9ed4d45dcb6e admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:32.292181 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a535e5-d8dd-48c6-b22c-9ed4d45dcb6e admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:32.296857 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599840c-9942-425f-9533-096cd6ceb829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:32.297706 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599840c-9942-425f-9533-096cd6ceb829 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:32.298213 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599840c-9942-425f-9533-096cd6ceb829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:32.298672 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599840c-9942-425f-9533-096cd6ceb829 admin admin] PATH_INFO: `/v3/projects/8a27b3e9643741e486fc931afee737bf/users/5491ed66365140b9910a10d618a404f8/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:32.313955 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d599840c-9942-425f-9533-096cd6ceb829 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a27b3e9643741e486fc931afee737bf, user_id=5491ed66365140b9910a10d618a404f8, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:32.316748 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d599840c-9942-425f-9533-096cd6ceb829 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:32.396474 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1077/2161] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:32:32 2022] PUT /identity/v3/projects/8a27b3e9643741e486fc931afee737bf/users/5491ed66365140b9910a10d618a404f8/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:32.401504 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c53387-e556-46b5-a04f-edd45e851286 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:32.401798 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c53387-e556-46b5-a04f-edd45e851286 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:32.402080 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c53387-e556-46b5-a04f-edd45e851286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:32.423994 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-74c53387-e556-46b5-a04f-edd45e851286 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:32.425334 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-74c53387-e556-46b5-a04f-edd45e851286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:32.474068 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:32.509195 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1085/2162] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:32.520109 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599840c-9942-425f-9533-096cd6ceb829 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:32.527822 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-d599840c-9942-425f-9533-096cd6ceb829 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:32.605591 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599840c-9942-425f-9533-096cd6ceb829 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:32.609846 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f1a69-2a32-49cf-a43c-f9efa92778f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:32.610563 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f1a69-2a32-49cf-a43c-f9efa92778f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:32.610756 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f1a69-2a32-49cf-a43c-f9efa92778f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:32.611049 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f1a69-2a32-49cf-a43c-f9efa92778f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:32.622126 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f1a69-2a32-49cf-a43c-f9efa92778f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:32.623689 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f1a69-2a32-49cf-a43c-f9efa92778f3 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:32.635824 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1078/2163] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:32:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:32:33.950434 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6a159-c1f2-4291-9600-5dfe5da3d980 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:33.950931 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6a159-c1f2-4291-9600-5dfe5da3d980 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:33.951060 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6a159-c1f2-4291-9600-5dfe5da3d980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:33.980036 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-98a6a159-c1f2-4291-9600-5dfe5da3d980 None None] MFA Rules not processed for user `5491ed66365140b9910a10d618a404f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:33.982737 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-98a6a159-c1f2-4291-9600-5dfe5da3d980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:34.049916 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1086/2164] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:34.054746 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f1a69-2a32-49cf-a43c-f9efa92778f3 service neutron] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:34.066817 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f1a69-2a32-49cf-a43c-f9efa92778f3 service neutron] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:34.071730 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879ad6d3-4e4d-420e-9314-19868d9bed15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:34.073479 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ad6d3-4e4d-420e-9314-19868d9bed15 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:34.073479 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ad6d3-4e4d-420e-9314-19868d9bed15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:34.073479 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ad6d3-4e4d-420e-9314-19868d9bed15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:34.073741 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879ad6d3-4e4d-420e-9314-19868d9bed15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:34.075593 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879ad6d3-4e4d-420e-9314-19868d9bed15 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:34.208566 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1079/2165] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:32:34 2022] POST /identity/v3/projects => generated 371 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:32:34.212691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a6a159-c1f2-4291-9600-5dfe5da3d980 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:34.223673 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a6a159-c1f2-4291-9600-5dfe5da3d980 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:34.280778 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bcbd8c-72ed-4633-8153-5b1897662661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:34.281408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bcbd8c-72ed-4633-8153-5b1897662661 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:34.281787 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bcbd8c-72ed-4633-8153-5b1897662661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:34.282066 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bcbd8c-72ed-4633-8153-5b1897662661 admin admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:34.282730 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bcbd8c-72ed-4633-8153-5b1897662661 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:34.284790 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bcbd8c-72ed-4633-8153-5b1897662661 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:34.410204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1087/2166] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:32:34 2022] POST /identity/v3/users => generated 292 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:32:34.416889 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879ad6d3-4e4d-420e-9314-19868d9bed15 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:34.432309 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879ad6d3-4e4d-420e-9314-19868d9bed15 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:34.437820 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5faea65-00ab-4261-a629-42a20c542588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:34.438452 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5faea65-00ab-4261-a629-42a20c542588 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:34.438738 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5faea65-00ab-4261-a629-42a20c542588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:34.439018 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5faea65-00ab-4261-a629-42a20c542588 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:34.439717 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5faea65-00ab-4261-a629-42a20c542588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:34.441369 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5faea65-00ab-4261-a629-42a20c542588 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:34.452498 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1080/2167] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:32:34.457272 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bcbd8c-72ed-4633-8153-5b1897662661 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:34.467783 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bcbd8c-72ed-4633-8153-5b1897662661 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:34.473335 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf13cf0-5dc9-4a65-8811-493ed64582b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:34.474744 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf13cf0-5dc9-4a65-8811-493ed64582b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:34.475053 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf13cf0-5dc9-4a65-8811-493ed64582b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:34.475343 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf13cf0-5dc9-4a65-8811-493ed64582b4 admin admin] PATH_INFO: `/v3/projects/190e2e9877414b9eb6bf1f141ea14711/users/24ed074740414ce8868720f7f55e6cda/roles/7c88c815db854954b516ab97cd471647` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:34.492705 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf13cf0-5dc9-4a65-8811-493ed64582b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=190e2e9877414b9eb6bf1f141ea14711, user_id=24ed074740414ce8868720f7f55e6cda, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:34.495391 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf13cf0-5dc9-4a65-8811-493ed64582b4 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:34.545103 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1088/2168] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:32:34 2022] PUT /identity/v3/projects/190e2e9877414b9eb6bf1f141ea14711/users/24ed074740414ce8868720f7f55e6cda/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:34.549914 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5faea65-00ab-4261-a629-42a20c542588 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:34.560289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5faea65-00ab-4261-a629-42a20c542588 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:34.620170 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b044b-dcb2-42f3-a851-3011f29b092a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:34.620793 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b044b-dcb2-42f3-a851-3011f29b092a admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:34.621193 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b044b-dcb2-42f3-a851-3011f29b092a admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:34.621471 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b044b-dcb2-42f3-a851-3011f29b092a admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:34.622161 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5b044b-dcb2-42f3-a851-3011f29b092a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:34.624457 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5b044b-dcb2-42f3-a851-3011f29b092a admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:34.634586 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1081/2169] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:34 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:32:34.639777 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf13cf0-5dc9-4a65-8811-493ed64582b4 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:34.651213 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf13cf0-5dc9-4a65-8811-493ed64582b4 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:34.658956 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a6f966-9b2b-4794-9251-6229aaa996ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:34.659636 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6f966-9b2b-4794-9251-6229aaa996ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:34.659921 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6f966-9b2b-4794-9251-6229aaa996ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:34.660256 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6f966-9b2b-4794-9251-6229aaa996ac admin admin] PATH_INFO: `/v3/projects/190e2e9877414b9eb6bf1f141ea14711/users/24ed074740414ce8868720f7f55e6cda/roles/7c88c815db854954b516ab97cd471647` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:34.663425 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a6f966-9b2b-4794-9251-6229aaa996ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=190e2e9877414b9eb6bf1f141ea14711, user_id=24ed074740414ce8868720f7f55e6cda, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:34.666208 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a6f966-9b2b-4794-9251-6229aaa996ac admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:34.677669 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1089/2170] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:32:34 2022] PUT /identity/v3/projects/190e2e9877414b9eb6bf1f141ea14711/users/24ed074740414ce8868720f7f55e6cda/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:35.899878 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f91ce-0660-4970-b897-97af209c5b8e None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:35.900438 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f91ce-0660-4970-b897-97af209c5b8e None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:35.900531 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f91ce-0660-4970-b897-97af209c5b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:35.930206 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-0b3f91ce-0660-4970-b897-97af209c5b8e None None] MFA Rules not processed for user `24ed074740414ce8868720f7f55e6cda`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:35.931690 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-0b3f91ce-0660-4970-b897-97af209c5b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:35.962001 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:36.003326 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1082/2171] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:35 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:36.014755 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a6f966-9b2b-4794-9251-6229aaa996ac admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:36.016563 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-92a6f966-9b2b-4794-9251-6229aaa996ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:36.047774 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:36.074487 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a6f966-9b2b-4794-9251-6229aaa996ac admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:36.078517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee3351d-4519-456e-a291-c4b4fd795967 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:36.079126 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee3351d-4519-456e-a291-c4b4fd795967 service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:36.079417 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee3351d-4519-456e-a291-c4b4fd795967 service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:36.079737 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee3351d-4519-456e-a291-c4b4fd795967 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:36.144919 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee3351d-4519-456e-a291-c4b4fd795967 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:36.146357 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee3351d-4519-456e-a291-c4b4fd795967 service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:36.155421 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1090/2172] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:32:36 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:32:37.861038 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3f91ce-0660-4970-b897-97af209c5b8e None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:37.870880 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3f91ce-0660-4970-b897-97af209c5b8e None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:37.874478 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c174fb18-8fe6-4040-9f5c-042dbcb8ec33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:37.875691 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174fb18-8fe6-4040-9f5c-042dbcb8ec33 service cinder] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:37.875973 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174fb18-8fe6-4040-9f5c-042dbcb8ec33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:37.876288 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174fb18-8fe6-4040-9f5c-042dbcb8ec33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:37.885468 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c174fb18-8fe6-4040-9f5c-042dbcb8ec33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:37.886905 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c174fb18-8fe6-4040-9f5c-042dbcb8ec33 service cinder] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:37.896985 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1083/2173] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:32:37 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:32:38.135437 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee3351d-4519-456e-a291-c4b4fd795967 service cinder] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:38.137019 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-4ee3351d-4519-456e-a291-c4b4fd795967 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:38.202026 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee3351d-4519-456e-a291-c4b4fd795967 service cinder] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:38.207622 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0645bb2-d828-437d-850e-bf71773e6a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:38.208333 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0645bb2-d828-437d-850e-bf71773e6a83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:38.208624 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0645bb2-d828-437d-850e-bf71773e6a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:38.208931 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0645bb2-d828-437d-850e-bf71773e6a83 admin admin] PATH_INFO: `/v3/users/4e6072ef3e1f435f86e3a0babc2903f9` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:38.210164 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0645bb2-d828-437d-850e-bf71773e6a83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e6072ef3e1f435f86e3a0babc2903f9)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:38.212154 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0645bb2-d828-437d-850e-bf71773e6a83 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:38.461445 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c0645bb2-d828-437d-850e-bf71773e6a83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e6072ef3e1f435f86e3a0babc2903f9'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:38.462163 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c0645bb2-d828-437d-850e-bf71773e6a83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e6072ef3e1f435f86e3a0babc2903f9'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:38.479343 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c0645bb2-d828-437d-850e-bf71773e6a83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e6072ef3e1f435f86e3a0babc2903f9'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:38.532396 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c0645bb2-d828-437d-850e-bf71773e6a83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e6072ef3e1f435f86e3a0babc2903f9'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:38.542117 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c0645bb2-d828-437d-850e-bf71773e6a83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e6072ef3e1f435f86e3a0babc2903f9'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:38.544728 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:38.550822 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1091/2174] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:32:38 2022] DELETE /identity/v3/users/4e6072ef3e1f435f86e3a0babc2903f9 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:38.555884 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b9118-1754-4cea-b01d-11c0b6ae727b None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:38.556262 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b9118-1754-4cea-b01d-11c0b6ae727b None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:38.556624 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b9118-1754-4cea-b01d-11c0b6ae727b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:38.577254 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-ec5b9118-1754-4cea-b01d-11c0b6ae727b None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:38.578631 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-ec5b9118-1754-4cea-b01d-11c0b6ae727b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:38.665333 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1084/2175] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:32:38.680005 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0645bb2-d828-437d-850e-bf71773e6a83 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:38.680545 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-c0645bb2-d828-437d-850e-bf71773e6a83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:38.738807 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0645bb2-d828-437d-850e-bf71773e6a83 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:38.742765 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4eea5-a29a-47e3-8914-f18ad14e94a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:38.743386 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4eea5-a29a-47e3-8914-f18ad14e94a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:38.743670 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4eea5-a29a-47e3-8914-f18ad14e94a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:38.743959 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4eea5-a29a-47e3-8914-f18ad14e94a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:38.754130 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e4eea5-a29a-47e3-8914-f18ad14e94a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:38.755571 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e4eea5-a29a-47e3-8914-f18ad14e94a9 service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:38.765883 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1092/2176] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:32:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:32:39.187358 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b9118-1754-4cea-b01d-11c0b6ae727b None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:39.187358 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-ec5b9118-1754-4cea-b01d-11c0b6ae727b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:39.206394 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b9118-1754-4cea-b01d-11c0b6ae727b None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:39.212093 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649bdb9c-c665-4f4b-81ad-33deb98b8f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:39.212839 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649bdb9c-c665-4f4b-81ad-33deb98b8f52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:39.213026 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649bdb9c-c665-4f4b-81ad-33deb98b8f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:39.213307 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649bdb9c-c665-4f4b-81ad-33deb98b8f52 admin admin] PATH_INFO: `/v3/projects/4780c1a68871410494f8e5631eab73df` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:39.214470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649bdb9c-c665-4f4b-81ad-33deb98b8f52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4780c1a68871410494f8e5631eab73df)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:39.216483 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649bdb9c-c665-4f4b-81ad-33deb98b8f52 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:39.338153 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-649bdb9c-c665-4f4b-81ad-33deb98b8f52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4780c1a68871410494f8e5631eab73df', 'request_id': 'req-649bdb9c-c665-4f4b-81ad-33deb98b8f52'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:39.344857 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-649bdb9c-c665-4f4b-81ad-33deb98b8f52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4780c1a68871410494f8e5631eab73df', 'request_id': 'req-649bdb9c-c665-4f4b-81ad-33deb98b8f52'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:39.440587 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-649bdb9c-c665-4f4b-81ad-33deb98b8f52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4780c1a68871410494f8e5631eab73df', 'request_id': 'req-649bdb9c-c665-4f4b-81ad-33deb98b8f52'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:39.454534 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-649bdb9c-c665-4f4b-81ad-33deb98b8f52 admin admin] The token cache is being invalidate because project 4780c1a68871410494f8e5631eab73df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:32:39.460268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-649bdb9c-c665-4f4b-81ad-33deb98b8f52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:39.460268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-649bdb9c-c665-4f4b-81ad-33deb98b8f52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:39.460268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1085/2177] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:32:39 2022] DELETE /identity/v3/projects/4780c1a68871410494f8e5631eab73df => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:39.461298 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4eea5-a29a-47e3-8914-f18ad14e94a9 service neutron] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:39.471762 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4eea5-a29a-47e3-8914-f18ad14e94a9 service neutron] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:39.542378 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b7db2-786f-4443-b46e-29a2adadbcfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:39.545175 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b7db2-786f-4443-b46e-29a2adadbcfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:39.545175 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b7db2-786f-4443-b46e-29a2adadbcfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:39.545175 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b7db2-786f-4443-b46e-29a2adadbcfc admin admin] PATH_INFO: `/v3/users/4562496d43b64fb3b2249a8c0148da72` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:39.545175 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9b7db2-786f-4443-b46e-29a2adadbcfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=4562496d43b64fb3b2249a8c0148da72)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:39.547898 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9b7db2-786f-4443-b46e-29a2adadbcfc admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:39.685380 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-aa9b7db2-786f-4443-b46e-29a2adadbcfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4562496d43b64fb3b2249a8c0148da72'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:39.685940 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-aa9b7db2-786f-4443-b46e-29a2adadbcfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4562496d43b64fb3b2249a8c0148da72'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:39.701103 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-aa9b7db2-786f-4443-b46e-29a2adadbcfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4562496d43b64fb3b2249a8c0148da72'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:39.707043 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649bdb9c-c665-4f4b-81ad-33deb98b8f52 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:39.708959 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-649bdb9c-c665-4f4b-81ad-33deb98b8f52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:39.722430 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649bdb9c-c665-4f4b-81ad-33deb98b8f52 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:39.727224 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37767659-dfbe-4e66-8688-7ed6454437b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:39.727880 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37767659-dfbe-4e66-8688-7ed6454437b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:39.728329 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37767659-dfbe-4e66-8688-7ed6454437b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:39.728617 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37767659-dfbe-4e66-8688-7ed6454437b2 admin admin] PATH_INFO: `/v3/users/459a32aeadda435c8b54bbb1ee7a63ef` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:39.732247 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37767659-dfbe-4e66-8688-7ed6454437b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=459a32aeadda435c8b54bbb1ee7a63ef)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:39.732247 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37767659-dfbe-4e66-8688-7ed6454437b2 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:39.772481 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-aa9b7db2-786f-4443-b46e-29a2adadbcfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4562496d43b64fb3b2249a8c0148da72'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:39.780431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-aa9b7db2-786f-4443-b46e-29a2adadbcfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4562496d43b64fb3b2249a8c0148da72'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:39.787254 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1093/2178] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:32:39 2022] DELETE /identity/v3/users/4562496d43b64fb3b2249a8c0148da72 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:39.912018 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:39.929758 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-37767659-dfbe-4e66-8688-7ed6454437b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '459a32aeadda435c8b54bbb1ee7a63ef'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:39.938027 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-37767659-dfbe-4e66-8688-7ed6454437b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '459a32aeadda435c8b54bbb1ee7a63ef'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:39.938734 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-37767659-dfbe-4e66-8688-7ed6454437b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '459a32aeadda435c8b54bbb1ee7a63ef'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:39.950440 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-37767659-dfbe-4e66-8688-7ed6454437b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '459a32aeadda435c8b54bbb1ee7a63ef'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:39.951089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-37767659-dfbe-4e66-8688-7ed6454437b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '459a32aeadda435c8b54bbb1ee7a63ef'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:40.059945 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1086/2179] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:32:39 2022] DELETE /identity/v3/users/459a32aeadda435c8b54bbb1ee7a63ef => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:32:40.065563 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697bd974-d1f6-4166-81c9-727aebdb608d None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:40.065954 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697bd974-d1f6-4166-81c9-727aebdb608d None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:40.066180 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697bd974-d1f6-4166-81c9-727aebdb608d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:40.086387 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-697bd974-d1f6-4166-81c9-727aebdb608d None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:40.087599 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-697bd974-d1f6-4166-81c9-727aebdb608d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:40.106905 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:40.161066 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1094/2180] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:40.171428 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37767659-dfbe-4e66-8688-7ed6454437b2 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:40.173174 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-37767659-dfbe-4e66-8688-7ed6454437b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:40.233727 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37767659-dfbe-4e66-8688-7ed6454437b2 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:40.237681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d157719f-8609-4c19-b1ff-e6b21b9d9c11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:40.238300 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d157719f-8609-4c19-b1ff-e6b21b9d9c11 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:40.238576 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d157719f-8609-4c19-b1ff-e6b21b9d9c11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:40.238849 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d157719f-8609-4c19-b1ff-e6b21b9d9c11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:40.248449 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d157719f-8609-4c19-b1ff-e6b21b9d9c11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:40.249950 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d157719f-8609-4c19-b1ff-e6b21b9d9c11 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:40.260302 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1087/2181] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:32:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:32:40.266119 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697bd974-d1f6-4166-81c9-727aebdb608d None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:40.267729 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-697bd974-d1f6-4166-81c9-727aebdb608d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:40.286397 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697bd974-d1f6-4166-81c9-727aebdb608d None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:40.293339 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ad52b-688e-4b94-965f-74abd80de757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:40.293339 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ad52b-688e-4b94-965f-74abd80de757 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:40.293339 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ad52b-688e-4b94-965f-74abd80de757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:40.293339 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ad52b-688e-4b94-965f-74abd80de757 admin admin] PATH_INFO: `/v3/projects/e076428ec4f2421693186611b1789c0b` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:40.293979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279ad52b-688e-4b94-965f-74abd80de757 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e076428ec4f2421693186611b1789c0b)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:40.296098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279ad52b-688e-4b94-965f-74abd80de757 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:40.430336 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-279ad52b-688e-4b94-965f-74abd80de757 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e076428ec4f2421693186611b1789c0b', 'request_id': 'req-279ad52b-688e-4b94-965f-74abd80de757'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:40.436690 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-279ad52b-688e-4b94-965f-74abd80de757 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e076428ec4f2421693186611b1789c0b', 'request_id': 'req-279ad52b-688e-4b94-965f-74abd80de757'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:40.442364 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-279ad52b-688e-4b94-965f-74abd80de757 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e076428ec4f2421693186611b1789c0b', 'request_id': 'req-279ad52b-688e-4b94-965f-74abd80de757'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:40.532809 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-279ad52b-688e-4b94-965f-74abd80de757 admin admin] The token cache is being invalidate because project e076428ec4f2421693186611b1789c0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:32:40.533499 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-279ad52b-688e-4b94-965f-74abd80de757 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:40.533917 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-279ad52b-688e-4b94-965f-74abd80de757 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:40.536081 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1095/2182] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:32:40 2022] DELETE /identity/v3/projects/e076428ec4f2421693186611b1789c0b => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:40.540410 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc975a-6c50-4865-a4b4-680dc6807a19 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:40.540706 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc975a-6c50-4865-a4b4-680dc6807a19 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:40.541019 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc975a-6c50-4865-a4b4-680dc6807a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:40.563587 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-21cc975a-6c50-4865-a4b4-680dc6807a19 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:40.564876 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-21cc975a-6c50-4865-a4b4-680dc6807a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:40.643359 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1088/2183] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:40.651877 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a155b-3d0f-48b9-8a4f-1974235bd811 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:40.652218 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a155b-3d0f-48b9-8a4f-1974235bd811 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:40.652537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a155b-3d0f-48b9-8a4f-1974235bd811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:40.674833 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-658a155b-3d0f-48b9-8a4f-1974235bd811 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:40.676523 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-658a155b-3d0f-48b9-8a4f-1974235bd811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:40.685785 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1096/2184] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:40.692015 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cc975a-6c50-4865-a4b4-680dc6807a19 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:40.705672 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cc975a-6c50-4865-a4b4-680dc6807a19 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:40.710756 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3798a-85e7-493a-9c97-a90a939d7401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:40.711390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3798a-85e7-493a-9c97-a90a939d7401 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:40.711670 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3798a-85e7-493a-9c97-a90a939d7401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:40.711948 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3798a-85e7-493a-9c97-a90a939d7401 admin admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:40.712717 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d3798a-85e7-493a-9c97-a90a939d7401 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:32:40.713024 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d3798a-85e7-493a-9c97-a90a939d7401 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:40.714647 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d3798a-85e7-493a-9c97-a90a939d7401 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:40.726598 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1089/2185] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:32:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:32:40.730981 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658a155b-3d0f-48b9-8a4f-1974235bd811 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:40.742337 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658a155b-3d0f-48b9-8a4f-1974235bd811 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:40.742985 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3798a-85e7-493a-9c97-a90a939d7401 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:40.745101 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-a8d3798a-85e7-493a-9c97-a90a939d7401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:40.748775 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82adeec7-3f2a-4c30-94b4-d4489687de65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:40.749458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82adeec7-3f2a-4c30-94b4-d4489687de65 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:40.750273 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82adeec7-3f2a-4c30-94b4-d4489687de65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:40.750556 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82adeec7-3f2a-4c30-94b4-d4489687de65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:40.751279 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82adeec7-3f2a-4c30-94b4-d4489687de65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:40.754636 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82adeec7-3f2a-4c30-94b4-d4489687de65 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:40.761125 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3798a-85e7-493a-9c97-a90a939d7401 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:40.769005 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1fe919-8573-4099-bce5-2f5d52db111d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:40.769902 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1fe919-8573-4099-bce5-2f5d52db111d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:40.770327 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1fe919-8573-4099-bce5-2f5d52db111d admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:40.770793 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1fe919-8573-4099-bce5-2f5d52db111d admin admin] PATH_INFO: `/v3/projects/c5bdf7d03cb745eba3d2a6865ba84766` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:40.772252 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1fe919-8573-4099-bce5-2f5d52db111d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5bdf7d03cb745eba3d2a6865ba84766)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:40.774926 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1fe919-8573-4099-bce5-2f5d52db111d admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:40.846682 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1097/2186] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:32:40 2022] POST /identity/v3/projects => generated 379 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:32:40.851531 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82adeec7-3f2a-4c30-94b4-d4489687de65 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:40.862423 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:40.878788 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-ee1fe919-8573-4099-bce5-2f5d52db111d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5bdf7d03cb745eba3d2a6865ba84766', 'request_id': 'req-ee1fe919-8573-4099-bce5-2f5d52db111d'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:40.886340 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-ee1fe919-8573-4099-bce5-2f5d52db111d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5bdf7d03cb745eba3d2a6865ba84766', 'request_id': 'req-ee1fe919-8573-4099-bce5-2f5d52db111d'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:40.930258 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82adeec7-3f2a-4c30-94b4-d4489687de65 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:40.961378 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-ee1fe919-8573-4099-bce5-2f5d52db111d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5bdf7d03cb745eba3d2a6865ba84766', 'request_id': 'req-ee1fe919-8573-4099-bce5-2f5d52db111d'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:40.967360 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:40.975921 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-ee1fe919-8573-4099-bce5-2f5d52db111d admin admin] The token cache is being invalidate because project c5bdf7d03cb745eba3d2a6865ba84766 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:32:40.976626 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-ee1fe919-8573-4099-bce5-2f5d52db111d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:40.977201 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-ee1fe919-8573-4099-bce5-2f5d52db111d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:40.980612 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1090/2187] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:32:40 2022] DELETE /identity/v3/projects/c5bdf7d03cb745eba3d2a6865ba84766 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:40.987901 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5d587-4a15-4385-84f9-d45f5285ebda None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:40.988317 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5d587-4a15-4385-84f9-d45f5285ebda None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:40.988601 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5d587-4a15-4385-84f9-d45f5285ebda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:41.013538 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-9cc5d587-4a15-4385-84f9-d45f5285ebda None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:41.017693 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-9cc5d587-4a15-4385-84f9-d45f5285ebda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:41.027860 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4f953-f39d-4e0a-afd7-a0c6c8e65762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:41.028525 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4f953-f39d-4e0a-afd7-a0c6c8e65762 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:41.028813 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4f953-f39d-4e0a-afd7-a0c6c8e65762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:41.029095 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4f953-f39d-4e0a-afd7-a0c6c8e65762 admin admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:41.030280 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e4f953-f39d-4e0a-afd7-a0c6c8e65762 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:41.033319 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e4f953-f39d-4e0a-afd7-a0c6c8e65762 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:41.089725 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1091/2188] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:41.097230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb85a0-c030-4f05-8cda-a25c48781f05 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:41.097519 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb85a0-c030-4f05-8cda-a25c48781f05 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:41.097797 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb85a0-c030-4f05-8cda-a25c48781f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:41.116769 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-20fb85a0-c030-4f05-8cda-a25c48781f05 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:41.118037 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-20fb85a0-c030-4f05-8cda-a25c48781f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:41.127273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1092/2189] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:32:41.132332 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fb85a0-c030-4f05-8cda-a25c48781f05 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:41.143026 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fb85a0-c030-4f05-8cda-a25c48781f05 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:41.148043 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6396f991-6c62-455b-8e65-fa439fee3840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:41.153705 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396f991-6c62-455b-8e65-fa439fee3840 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:41.153705 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396f991-6c62-455b-8e65-fa439fee3840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:41.153705 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396f991-6c62-455b-8e65-fa439fee3840 admin admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:41.153705 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6396f991-6c62-455b-8e65-fa439fee3840 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:32:41.153705 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6396f991-6c62-455b-8e65-fa439fee3840 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:41.153705 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6396f991-6c62-455b-8e65-fa439fee3840 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:41.165809 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1098/2190] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:32:40 2022] POST /identity/v3/users => generated 290 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:32:41.167464 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4f953-f39d-4e0a-afd7-a0c6c8e65762 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:41.172140 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1093/2191] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:32:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:32:41.173758 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-45e4f953-f39d-4e0a-afd7-a0c6c8e65762 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:41.174435 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6396f991-6c62-455b-8e65-fa439fee3840 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:41.186282 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6396f991-6c62-455b-8e65-fa439fee3840 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:41.186557 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4f953-f39d-4e0a-afd7-a0c6c8e65762 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:41.192190 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c13027-5327-49eb-ad63-9bc02740649e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:41.193021 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c13027-5327-49eb-ad63-9bc02740649e admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:41.193665 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c13027-5327-49eb-ad63-9bc02740649e admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:41.194025 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c13027-5327-49eb-ad63-9bc02740649e admin admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:41.194790 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c13027-5327-49eb-ad63-9bc02740649e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:41.195200 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc61bb5-8444-4363-aef3-5ad86058d734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:41.195991 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc61bb5-8444-4363-aef3-5ad86058d734 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:41.196324 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc61bb5-8444-4363-aef3-5ad86058d734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:41.196838 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c13027-5327-49eb-ad63-9bc02740649e admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:41.197117 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc61bb5-8444-4363-aef3-5ad86058d734 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:41.197883 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc61bb5-8444-4363-aef3-5ad86058d734 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:41.200148 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc61bb5-8444-4363-aef3-5ad86058d734 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:41.214710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1099/2192] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 15:32:41.215661 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc61bb5-8444-4363-aef3-5ad86058d734 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:41.226465 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc61bb5-8444-4363-aef3-5ad86058d734 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:41.234851 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa09fe-e7b2-4bcd-8b7d-cf1c00cb2e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:41.235507 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa09fe-e7b2-4bcd-8b7d-cf1c00cb2e1a admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:41.235796 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa09fe-e7b2-4bcd-8b7d-cf1c00cb2e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:41.236157 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa09fe-e7b2-4bcd-8b7d-cf1c00cb2e1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:41.236880 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afa09fe-e7b2-4bcd-8b7d-cf1c00cb2e1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:41.238504 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afa09fe-e7b2-4bcd-8b7d-cf1c00cb2e1a admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:41.250932 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1100/2193] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 15:32:41.260156 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa09fe-e7b2-4bcd-8b7d-cf1c00cb2e1a admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:41.270563 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa09fe-e7b2-4bcd-8b7d-cf1c00cb2e1a admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:41.275479 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09d34a-00ca-49da-ba33-8d20c2f1034d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:41.276170 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09d34a-00ca-49da-ba33-8d20c2f1034d admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:41.276695 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09d34a-00ca-49da-ba33-8d20c2f1034d admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:41.276986 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09d34a-00ca-49da-ba33-8d20c2f1034d admin admin] PATH_INFO: `/v3/projects/70fc25acb0b348ed82d67553e5aa6ce2/users/50f1d5a446a34b3f84659ec3f851df86/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:41.292714 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a09d34a-00ca-49da-ba33-8d20c2f1034d admin admin] RBAC: Authorizing `identity:create_grant(project_id=70fc25acb0b348ed82d67553e5aa6ce2, user_id=50f1d5a446a34b3f84659ec3f851df86, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:41.295697 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a09d34a-00ca-49da-ba33-8d20c2f1034d admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:41.327981 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1094/2194] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:32:41 2022] POST /identity/v3/projects => generated 379 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:32:41.332057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c13027-5327-49eb-ad63-9bc02740649e admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:41.393473 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c13027-5327-49eb-ad63-9bc02740649e admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:41.396938 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1101/2195] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:32:41 2022] PUT /identity/v3/projects/70fc25acb0b348ed82d67553e5aa6ce2/users/50f1d5a446a34b3f84659ec3f851df86/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:41.407513 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ceeab8-b9c3-48a8-9222-a14c39147014 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:41.407806 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ceeab8-b9c3-48a8-9222-a14c39147014 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:41.408111 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ceeab8-b9c3-48a8-9222-a14c39147014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:41.419445 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:41.436077 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-92ceeab8-b9c3-48a8-9222-a14c39147014 None None] MFA Rules not processed for user `50f1d5a446a34b3f84659ec3f851df86`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:41.437309 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-92ceeab8-b9c3-48a8-9222-a14c39147014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:41.456150 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15787298-d0de-4676-a1ad-04eee79d1287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:41.456150 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15787298-d0de-4676-a1ad-04eee79d1287 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:41.456150 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15787298-d0de-4676-a1ad-04eee79d1287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:41.456150 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15787298-d0de-4676-a1ad-04eee79d1287 admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:41.456833 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15787298-d0de-4676-a1ad-04eee79d1287 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:41.456833 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15787298-d0de-4676-a1ad-04eee79d1287 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:41.518226 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1102/2196] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:32:41.522687 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ceeab8-b9c3-48a8-9222-a14c39147014 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:41.533516 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ceeab8-b9c3-48a8-9222-a14c39147014 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:41.538167 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39058cc5-25d7-4d0a-99b7-aba5e59594f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:41.538761 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39058cc5-25d7-4d0a-99b7-aba5e59594f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:41.539051 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39058cc5-25d7-4d0a-99b7-aba5e59594f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:41.539332 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39058cc5-25d7-4d0a-99b7-aba5e59594f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:41.540028 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39058cc5-25d7-4d0a-99b7-aba5e59594f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:41.542007 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39058cc5-25d7-4d0a-99b7-aba5e59594f1 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:41.563976 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1095/2197] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:32:41 2022] POST /identity/v3/users => generated 290 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:32:41.565099 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15787298-d0de-4676-a1ad-04eee79d1287 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:41.575095 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15787298-d0de-4676-a1ad-04eee79d1287 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:41.580542 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e69a0e-ab85-4fef-aa28-5edf6342e298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:41.581149 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e69a0e-ab85-4fef-aa28-5edf6342e298 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:41.581433 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e69a0e-ab85-4fef-aa28-5edf6342e298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:41.581799 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e69a0e-ab85-4fef-aa28-5edf6342e298 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:41.582492 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e69a0e-ab85-4fef-aa28-5edf6342e298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:41.584651 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e69a0e-ab85-4fef-aa28-5edf6342e298 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:41.596103 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1096/2198] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 15:32:41.600161 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e69a0e-ab85-4fef-aa28-5edf6342e298 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:41.611320 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e69a0e-ab85-4fef-aa28-5edf6342e298 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:41.615656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1103/2199] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:32:41 2022] POST /identity/v3/projects => generated 379 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:32:41.619249 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cedcc8-279e-40e3-b847-d5e4acdfd724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:41.619249 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cedcc8-279e-40e3-b847-d5e4acdfd724 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:41.619249 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cedcc8-279e-40e3-b847-d5e4acdfd724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:41.619249 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cedcc8-279e-40e3-b847-d5e4acdfd724 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:41.619637 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cedcc8-279e-40e3-b847-d5e4acdfd724 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:41.621280 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cedcc8-279e-40e3-b847-d5e4acdfd724 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:41.622050 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39058cc5-25d7-4d0a-99b7-aba5e59594f1 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:41.634393 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1097/2200] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 15:32:41.639695 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cedcc8-279e-40e3-b847-d5e4acdfd724 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:41.650970 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cedcc8-279e-40e3-b847-d5e4acdfd724 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:41.670166 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:41.700871 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39058cc5-25d7-4d0a-99b7-aba5e59594f1 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:41.712371 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7e2fb1-16fa-4b64-a6d9-b70adad702ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:41.712930 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e2fb1-16fa-4b64-a6d9-b70adad702ff admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:41.713213 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e2fb1-16fa-4b64-a6d9-b70adad702ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:41.713747 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e2fb1-16fa-4b64-a6d9-b70adad702ff admin admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:41.714214 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7e2fb1-16fa-4b64-a6d9-b70adad702ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:41.716207 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7e2fb1-16fa-4b64-a6d9-b70adad702ff admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:41.717112 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe5a00-b496-4554-a4fe-284de9dc6092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:41.717782 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe5a00-b496-4554-a4fe-284de9dc6092 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:41.718072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe5a00-b496-4554-a4fe-284de9dc6092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:41.718420 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe5a00-b496-4554-a4fe-284de9dc6092 admin admin] PATH_INFO: `/v3/projects/99b26247f85d45c0a7c1e763758f2e69/users/1782b4e84c9f484297e17c248738105c/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:41.732821 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fe5a00-b496-4554-a4fe-284de9dc6092 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99b26247f85d45c0a7c1e763758f2e69, user_id=1782b4e84c9f484297e17c248738105c, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:41.736043 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fe5a00-b496-4554-a4fe-284de9dc6092 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:41.796157 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1104/2201] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:32:41 2022] POST /identity/v3/users => generated 290 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:32:41.800621 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7e2fb1-16fa-4b64-a6d9-b70adad702ff admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:41.811183 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7e2fb1-16fa-4b64-a6d9-b70adad702ff admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:41.813354 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1098/2202] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:32:41 2022] PUT /identity/v3/projects/99b26247f85d45c0a7c1e763758f2e69/users/1782b4e84c9f484297e17c248738105c/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:41.819953 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69a232-c6c7-4208-97f3-884cc04bc391 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:41.820366 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69a232-c6c7-4208-97f3-884cc04bc391 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:41.820750 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69a232-c6c7-4208-97f3-884cc04bc391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:41.840564 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-eb69a232-c6c7-4208-97f3-884cc04bc391 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:41.841947 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-eb69a232-c6c7-4208-97f3-884cc04bc391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:41.873292 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174098ba-ad30-4e2b-b28a-1d1ddc9def5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:41.880126 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174098ba-ad30-4e2b-b28a-1d1ddc9def5d admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:41.880126 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174098ba-ad30-4e2b-b28a-1d1ddc9def5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:41.880126 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174098ba-ad30-4e2b-b28a-1d1ddc9def5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:41.880126 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174098ba-ad30-4e2b-b28a-1d1ddc9def5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:41.880126 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174098ba-ad30-4e2b-b28a-1d1ddc9def5d admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:41.891993 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1105/2203] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 15:32:41.893366 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174098ba-ad30-4e2b-b28a-1d1ddc9def5d admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:41.902966 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174098ba-ad30-4e2b-b28a-1d1ddc9def5d admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:41.908773 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6880193d-92ff-4067-9d50-2fe074b19edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:41.909238 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880193d-92ff-4067-9d50-2fe074b19edc admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:41.909548 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880193d-92ff-4067-9d50-2fe074b19edc admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:41.909830 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880193d-92ff-4067-9d50-2fe074b19edc admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:41.910223 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1099/2204] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:32:41.910534 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6880193d-92ff-4067-9d50-2fe074b19edc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:41.912159 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6880193d-92ff-4067-9d50-2fe074b19edc admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:41.920011 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb69a232-c6c7-4208-97f3-884cc04bc391 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:41.921519 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-eb69a232-c6c7-4208-97f3-884cc04bc391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:41.922725 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1106/2205] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:32:41.926662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6880193d-92ff-4067-9d50-2fe074b19edc admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:41.937574 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6880193d-92ff-4067-9d50-2fe074b19edc admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:41.942555 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050e559a-1fb7-4c70-8348-182dfc8559cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:41.943222 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e559a-1fb7-4c70-8348-182dfc8559cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:41.943514 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e559a-1fb7-4c70-8348-182dfc8559cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:41.943816 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e559a-1fb7-4c70-8348-182dfc8559cf admin admin] PATH_INFO: `/v3/projects/ff67433a68ef49b0b243549a13e8bffa/users/4232dfceee854234b309b18edb0e20d6/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:41.947913 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:41.960096 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050e559a-1fb7-4c70-8348-182dfc8559cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff67433a68ef49b0b243549a13e8bffa, user_id=4232dfceee854234b309b18edb0e20d6, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:41.963036 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050e559a-1fb7-4c70-8348-182dfc8559cf admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:41.987943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb69a232-c6c7-4208-97f3-884cc04bc391 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:41.991857 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f012fc0-724f-4a28-a5cd-cd30c409365e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:41.992513 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f012fc0-724f-4a28-a5cd-cd30c409365e service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:41.992798 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f012fc0-724f-4a28-a5cd-cd30c409365e service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:41.993075 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f012fc0-724f-4a28-a5cd-cd30c409365e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:42.002660 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f012fc0-724f-4a28-a5cd-cd30c409365e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:42.004151 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f012fc0-724f-4a28-a5cd-cd30c409365e service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:42.014059 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1100/2206] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:32:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 15:32:42.098672 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1107/2207] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:32:41 2022] PUT /identity/v3/projects/ff67433a68ef49b0b243549a13e8bffa/users/4232dfceee854234b309b18edb0e20d6/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:42.110313 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b168d491-3fb3-493b-a5d4-90be0dedaa30 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:42.110601 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b168d491-3fb3-493b-a5d4-90be0dedaa30 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:42.110886 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b168d491-3fb3-493b-a5d4-90be0dedaa30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:42.148725 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-b168d491-3fb3-493b-a5d4-90be0dedaa30 None None] MFA Rules not processed for user `4232dfceee854234b309b18edb0e20d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:42.150163 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-b168d491-3fb3-493b-a5d4-90be0dedaa30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:42.242243 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1101/2208] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:42.243432 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050e559a-1fb7-4c70-8348-182dfc8559cf admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:42.322839 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050e559a-1fb7-4c70-8348-182dfc8559cf admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:42.328576 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e329078-4aa4-4d55-9c31-dd028c625c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:42.329259 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e329078-4aa4-4d55-9c31-dd028c625c80 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:42.329479 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e329078-4aa4-4d55-9c31-dd028c625c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:42.329765 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e329078-4aa4-4d55-9c31-dd028c625c80 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:42.330436 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e329078-4aa4-4d55-9c31-dd028c625c80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:42.332403 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e329078-4aa4-4d55-9c31-dd028c625c80 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:42.410215 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1108/2209] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:32:42 2022] POST /identity/v3/projects => generated 379 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:32:42.414305 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b168d491-3fb3-493b-a5d4-90be0dedaa30 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:42.425883 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b168d491-3fb3-493b-a5d4-90be0dedaa30 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:42.488417 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b38b601-9bed-46ff-9add-c4663fb5d7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:42.489153 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b38b601-9bed-46ff-9add-c4663fb5d7df admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:42.489699 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b38b601-9bed-46ff-9add-c4663fb5d7df admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:42.489979 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b38b601-9bed-46ff-9add-c4663fb5d7df admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:42.490644 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b38b601-9bed-46ff-9add-c4663fb5d7df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:42.493077 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b38b601-9bed-46ff-9add-c4663fb5d7df admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:42.611454 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1102/2210] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:32:42 2022] POST /identity/v3/users => generated 296 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:32:42.617929 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e329078-4aa4-4d55-9c31-dd028c625c80 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:42.632437 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e329078-4aa4-4d55-9c31-dd028c625c80 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:42.638729 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b5456d-cc8f-48f8-956b-3266ffd7b71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:42.639580 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5456d-cc8f-48f8-956b-3266ffd7b71f admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:42.639824 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5456d-cc8f-48f8-956b-3266ffd7b71f admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:42.640137 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5456d-cc8f-48f8-956b-3266ffd7b71f admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:42.641233 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b5456d-cc8f-48f8-956b-3266ffd7b71f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:42.642549 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b5456d-cc8f-48f8-956b-3266ffd7b71f admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:42.648950 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:42.655404 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1109/2211] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:32:42.660331 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b38b601-9bed-46ff-9add-c4663fb5d7df admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:42.676630 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b38b601-9bed-46ff-9add-c4663fb5d7df admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:42.681939 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440fc670-3750-483c-bfe8-4fd1fe5c8e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:42.682846 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440fc670-3750-483c-bfe8-4fd1fe5c8e89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:42.683284 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440fc670-3750-483c-bfe8-4fd1fe5c8e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:42.683707 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440fc670-3750-483c-bfe8-4fd1fe5c8e89 admin admin] PATH_INFO: `/v3/projects/f150318010e146b3a2d4390e2709d1d2/users/4c081d957fff49d98e259e14d054666f/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:42.708555 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440fc670-3750-483c-bfe8-4fd1fe5c8e89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f150318010e146b3a2d4390e2709d1d2, user_id=4c081d957fff49d98e259e14d054666f, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:42.712308 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440fc670-3750-483c-bfe8-4fd1fe5c8e89 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:42.790458 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1103/2212] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:32:42 2022] PUT /identity/v3/projects/f150318010e146b3a2d4390e2709d1d2/users/4c081d957fff49d98e259e14d054666f/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:42.794853 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b5456d-cc8f-48f8-956b-3266ffd7b71f admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:42.813089 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b5456d-cc8f-48f8-956b-3266ffd7b71f admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:42.886749 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64188c39-c391-4315-b7ea-bdbe53f0de2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:42.887483 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64188c39-c391-4315-b7ea-bdbe53f0de2e admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:42.887864 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64188c39-c391-4315-b7ea-bdbe53f0de2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:42.888295 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64188c39-c391-4315-b7ea-bdbe53f0de2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:42.889080 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64188c39-c391-4315-b7ea-bdbe53f0de2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:42.895677 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64188c39-c391-4315-b7ea-bdbe53f0de2e admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:42.902551 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1110/2213] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:32:42.907358 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440fc670-3750-483c-bfe8-4fd1fe5c8e89 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:42.917811 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440fc670-3750-483c-bfe8-4fd1fe5c8e89 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:42.923833 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bab3c9-5e7b-4982-b8be-182f4779fa77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:42.924609 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bab3c9-5e7b-4982-b8be-182f4779fa77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:42.925394 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bab3c9-5e7b-4982-b8be-182f4779fa77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:42.925617 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bab3c9-5e7b-4982-b8be-182f4779fa77 admin admin] PATH_INFO: `/v3/projects/f150318010e146b3a2d4390e2709d1d2/users/4c081d957fff49d98e259e14d054666f/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:42.928212 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bab3c9-5e7b-4982-b8be-182f4779fa77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f150318010e146b3a2d4390e2709d1d2, user_id=4c081d957fff49d98e259e14d054666f, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:42.931006 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bab3c9-5e7b-4982-b8be-182f4779fa77 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:42.942625 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1104/2214] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:32:42 2022] PUT /identity/v3/projects/f150318010e146b3a2d4390e2709d1d2/users/4c081d957fff49d98e259e14d054666f/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:42.953655 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45933b-c38d-4b01-a001-47d90d30449d None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:42.953976 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45933b-c38d-4b01-a001-47d90d30449d None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:42.954259 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45933b-c38d-4b01-a001-47d90d30449d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:42.986344 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-9a45933b-c38d-4b01-a001-47d90d30449d None None] MFA Rules not processed for user `4c081d957fff49d98e259e14d054666f`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:42.986479 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-9a45933b-c38d-4b01-a001-47d90d30449d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:43.061189 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1111/2215] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:42 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:43.073333 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bab3c9-5e7b-4982-b8be-182f4779fa77 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:43.082882 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-a6bab3c9-5e7b-4982-b8be-182f4779fa77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:43.088724 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:43.158675 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bab3c9-5e7b-4982-b8be-182f4779fa77 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:43.162565 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea3243c-a55c-40a3-811c-a68cf2322ee0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:43.163362 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3243c-a55c-40a3-811c-a68cf2322ee0 service cinder] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:43.164160 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3243c-a55c-40a3-811c-a68cf2322ee0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:43.164160 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3243c-a55c-40a3-811c-a68cf2322ee0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:43.186202 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea3243c-a55c-40a3-811c-a68cf2322ee0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:43.187811 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea3243c-a55c-40a3-811c-a68cf2322ee0 service cinder] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:43.198071 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1105/2216] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:32:43 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:32:43.336315 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45933b-c38d-4b01-a001-47d90d30449d None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:43.337757 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-9a45933b-c38d-4b01-a001-47d90d30449d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:43.350201 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45933b-c38d-4b01-a001-47d90d30449d None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:43.354153 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb69c2-65a8-4fb4-9ca5-91073223a2af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:43.354734 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb69c2-65a8-4fb4-9ca5-91073223a2af service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:43.355051 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb69c2-65a8-4fb4-9ca5-91073223a2af service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:43.355332 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb69c2-65a8-4fb4-9ca5-91073223a2af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:43.427171 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb69c2-65a8-4fb4-9ca5-91073223a2af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:43.428872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb69c2-65a8-4fb4-9ca5-91073223a2af service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:43.440288 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1112/2217] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:32:43 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:32:48.235726 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea3243c-a55c-40a3-811c-a68cf2322ee0 service cinder] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:48.338577 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea3243c-a55c-40a3-811c-a68cf2322ee0 service cinder] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:48.344279 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0615b316-3f5d-424b-a0e1-13e7883d57db None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:48.344785 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784ad2c9-211f-4bc0-9cbb-54b32421b68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:48.344899 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0615b316-3f5d-424b-a0e1-13e7883d57db None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:48.344899 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0615b316-3f5d-424b-a0e1-13e7883d57db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:48.346714 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ad2c9-211f-4bc0-9cbb-54b32421b68f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:48.346714 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ad2c9-211f-4bc0-9cbb-54b32421b68f admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:48.346714 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ad2c9-211f-4bc0-9cbb-54b32421b68f admin admin] PATH_INFO: `/v3/users/50f1d5a446a34b3f84659ec3f851df86` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:48.347726 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784ad2c9-211f-4bc0-9cbb-54b32421b68f admin admin] RBAC: Authorizing `identity:delete_user(user_id=50f1d5a446a34b3f84659ec3f851df86)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:48.349970 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784ad2c9-211f-4bc0-9cbb-54b32421b68f admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:48.401404 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-0615b316-3f5d-424b-a0e1-13e7883d57db None None] MFA Rules not processed for user `1782b4e84c9f484297e17c248738105c`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:48.402717 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-0615b316-3f5d-424b-a0e1-13e7883d57db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:48.416730 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:48.479387 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1113/2218] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:48.484547 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0615b316-3f5d-424b-a0e1-13e7883d57db None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:48.495985 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0615b316-3f5d-424b-a0e1-13e7883d57db None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:48.501288 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d23bad-5c5b-4f24-b2aa-b59f86234e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:48.502072 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d23bad-5c5b-4f24-b2aa-b59f86234e8d admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:48.502358 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d23bad-5c5b-4f24-b2aa-b59f86234e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:48.502643 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d23bad-5c5b-4f24-b2aa-b59f86234e8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:48.503328 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d23bad-5c5b-4f24-b2aa-b59f86234e8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:48.505299 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d23bad-5c5b-4f24-b2aa-b59f86234e8d admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:48.553278 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-784ad2c9-211f-4bc0-9cbb-54b32421b68f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50f1d5a446a34b3f84659ec3f851df86'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:48.564031 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-784ad2c9-211f-4bc0-9cbb-54b32421b68f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50f1d5a446a34b3f84659ec3f851df86'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:48.564922 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-784ad2c9-211f-4bc0-9cbb-54b32421b68f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50f1d5a446a34b3f84659ec3f851df86'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:48.581933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-784ad2c9-211f-4bc0-9cbb-54b32421b68f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50f1d5a446a34b3f84659ec3f851df86'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:48.583367 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-784ad2c9-211f-4bc0-9cbb-54b32421b68f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50f1d5a446a34b3f84659ec3f851df86'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:48.609259 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1114/2219] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:32:48 2022] POST /identity/v3/projects => generated 377 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:32:48.612959 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d23bad-5c5b-4f24-b2aa-b59f86234e8d admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:48.615540 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-04d23bad-5c5b-4f24-b2aa-b59f86234e8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:48.665329 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1106/2220] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:32:48 2022] DELETE /identity/v3/users/50f1d5a446a34b3f84659ec3f851df86 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:48.670679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9267c12-132a-4ccf-a263-cf02379d1b55 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:48.671095 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9267c12-132a-4ccf-a263-cf02379d1b55 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:48.682592 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9267c12-132a-4ccf-a263-cf02379d1b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:48.693037 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-c9267c12-132a-4ccf-a263-cf02379d1b55 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:48.694333 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-c9267c12-132a-4ccf-a263-cf02379d1b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:48.699064 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d23bad-5c5b-4f24-b2aa-b59f86234e8d admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:48.722684 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:48.772639 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6be2f1-912d-4f70-a388-7561866a2885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:48.773547 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6be2f1-912d-4f70-a388-7561866a2885 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:48.773928 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6be2f1-912d-4f70-a388-7561866a2885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:48.774560 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6be2f1-912d-4f70-a388-7561866a2885 admin admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:48.775276 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6be2f1-912d-4f70-a388-7561866a2885 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:48.777653 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6be2f1-912d-4f70-a388-7561866a2885 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:48.789012 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1107/2221] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:48.798892 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9267c12-132a-4ccf-a263-cf02379d1b55 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:48.800669 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-c9267c12-132a-4ccf-a263-cf02379d1b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:48.864105 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1115/2222] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:32:48 2022] POST /identity/v3/users => generated 295 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:32:48.868182 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6be2f1-912d-4f70-a388-7561866a2885 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:48.871537 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9267c12-132a-4ccf-a263-cf02379d1b55 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:48.878464 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5663f9ad-1ccf-4571-a6b3-efc12754d8b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:48.879101 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663f9ad-1ccf-4571-a6b3-efc12754d8b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:48.879383 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663f9ad-1ccf-4571-a6b3-efc12754d8b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:48.879798 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663f9ad-1ccf-4571-a6b3-efc12754d8b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:48.882160 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6be2f1-912d-4f70-a388-7561866a2885 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:48.887966 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7723fa56-8833-45e2-9b3f-ca44fc1ee9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:48.888851 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723fa56-8833-45e2-9b3f-ca44fc1ee9ee admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:48.888974 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723fa56-8833-45e2-9b3f-ca44fc1ee9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:48.889173 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723fa56-8833-45e2-9b3f-ca44fc1ee9ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:48.901060 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7723fa56-8833-45e2-9b3f-ca44fc1ee9ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:48.901932 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5663f9ad-1ccf-4571-a6b3-efc12754d8b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:48.902031 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7723fa56-8833-45e2-9b3f-ca44fc1ee9ee admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:48.902129 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5663f9ad-1ccf-4571-a6b3-efc12754d8b9 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:48.910719 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1108/2223] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:32:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 15:32:48.915828 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1116/2224] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 15:32:48.916816 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5663f9ad-1ccf-4571-a6b3-efc12754d8b9 service neutron] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:48.928616 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5663f9ad-1ccf-4571-a6b3-efc12754d8b9 service neutron] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:48.939507 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebff801-8390-48e1-babb-63eecb874ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:48.940234 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebff801-8390-48e1-babb-63eecb874ee7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:48.940526 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebff801-8390-48e1-babb-63eecb874ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:48.940817 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebff801-8390-48e1-babb-63eecb874ee7 admin admin] PATH_INFO: `/v3/projects/b7974d0a036b4a8ba4478ba49790bf42/users/d62b8944fe534f10b71e305315a75f0f/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:48.963853 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebff801-8390-48e1-babb-63eecb874ee7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7974d0a036b4a8ba4478ba49790bf42, user_id=d62b8944fe534f10b71e305315a75f0f, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:48.968585 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebff801-8390-48e1-babb-63eecb874ee7 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:49.051684 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1109/2225] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:32:48 2022] PUT /identity/v3/projects/b7974d0a036b4a8ba4478ba49790bf42/users/d62b8944fe534f10b71e305315a75f0f/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:32:49.056327 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7723fa56-8833-45e2-9b3f-ca44fc1ee9ee admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:49.074937 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7723fa56-8833-45e2-9b3f-ca44fc1ee9ee admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:49.089919 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:49.150918 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8545361-27f4-4a07-8deb-087bf4a7e233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:49.152778 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8545361-27f4-4a07-8deb-087bf4a7e233 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:49.153119 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8545361-27f4-4a07-8deb-087bf4a7e233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:49.153410 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8545361-27f4-4a07-8deb-087bf4a7e233 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:49.154125 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8545361-27f4-4a07-8deb-087bf4a7e233 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:49.156906 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8545361-27f4-4a07-8deb-087bf4a7e233 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:49.187729 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1117/2226] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:32:49.191700 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebff801-8390-48e1-babb-63eecb874ee7 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:49.211127 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebff801-8390-48e1-babb-63eecb874ee7 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:49.248626 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8802fe62-dca9-444e-a304-d09298b08aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:49.248626 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802fe62-dca9-444e-a304-d09298b08aaa admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:49.248626 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802fe62-dca9-444e-a304-d09298b08aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:49.248626 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802fe62-dca9-444e-a304-d09298b08aaa admin admin] PATH_INFO: `/v3/projects/b7974d0a036b4a8ba4478ba49790bf42/users/d62b8944fe534f10b71e305315a75f0f/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:49.255255 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8802fe62-dca9-444e-a304-d09298b08aaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7974d0a036b4a8ba4478ba49790bf42, user_id=d62b8944fe534f10b71e305315a75f0f, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:49.259684 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8802fe62-dca9-444e-a304-d09298b08aaa admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:49.297427 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1110/2227] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:32:49 2022] PUT /identity/v3/projects/b7974d0a036b4a8ba4478ba49790bf42/users/d62b8944fe534f10b71e305315a75f0f/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:49.540201 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8545361-27f4-4a07-8deb-087bf4a7e233 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:49.542750 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-a8545361-27f4-4a07-8deb-087bf4a7e233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:49.566714 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8545361-27f4-4a07-8deb-087bf4a7e233 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:49.676942 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a35967-5b6a-4ebb-b4e3-b203a093d913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:49.677537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a35967-5b6a-4ebb-b4e3-b203a093d913 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:49.677825 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a35967-5b6a-4ebb-b4e3-b203a093d913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:49.678105 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a35967-5b6a-4ebb-b4e3-b203a093d913 admin admin] PATH_INFO: `/v3/projects/70fc25acb0b348ed82d67553e5aa6ce2` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:49.679497 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a35967-5b6a-4ebb-b4e3-b203a093d913 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70fc25acb0b348ed82d67553e5aa6ce2)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:49.684370 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a35967-5b6a-4ebb-b4e3-b203a093d913 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:49.821840 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-02a35967-5b6a-4ebb-b4e3-b203a093d913 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70fc25acb0b348ed82d67553e5aa6ce2', 'request_id': 'req-02a35967-5b6a-4ebb-b4e3-b203a093d913'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:49.829781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-02a35967-5b6a-4ebb-b4e3-b203a093d913 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70fc25acb0b348ed82d67553e5aa6ce2', 'request_id': 'req-02a35967-5b6a-4ebb-b4e3-b203a093d913'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:49.838181 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-02a35967-5b6a-4ebb-b4e3-b203a093d913 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70fc25acb0b348ed82d67553e5aa6ce2', 'request_id': 'req-02a35967-5b6a-4ebb-b4e3-b203a093d913'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:49.968818 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-02a35967-5b6a-4ebb-b4e3-b203a093d913 admin admin] The token cache is being invalidate because project 70fc25acb0b348ed82d67553e5aa6ce2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:32:49.969416 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-02a35967-5b6a-4ebb-b4e3-b203a093d913 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:49.970318 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-02a35967-5b6a-4ebb-b4e3-b203a093d913 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:49.975402 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1118/2228] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:32:49 2022] DELETE /identity/v3/projects/70fc25acb0b348ed82d67553e5aa6ce2 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:49.976717 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8802fe62-dca9-444e-a304-d09298b08aaa admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:49.978525 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-8802fe62-dca9-444e-a304-d09298b08aaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:49.998795 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8802fe62-dca9-444e-a304-d09298b08aaa admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:50.136204 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ffc3d7-a3f9-46bc-8927-8a783685c88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:50.137347 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffc3d7-a3f9-46bc-8927-8a783685c88b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:50.137805 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffc3d7-a3f9-46bc-8927-8a783685c88b admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:50.138222 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffc3d7-a3f9-46bc-8927-8a783685c88b admin admin] PATH_INFO: `/v3/users/4232dfceee854234b309b18edb0e20d6` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:50.140618 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ffc3d7-a3f9-46bc-8927-8a783685c88b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4232dfceee854234b309b18edb0e20d6)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:50.143719 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ffc3d7-a3f9-46bc-8927-8a783685c88b admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:50.329967 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:50.361335 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-84ffc3d7-a3f9-46bc-8927-8a783685c88b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4232dfceee854234b309b18edb0e20d6'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:50.371677 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-84ffc3d7-a3f9-46bc-8927-8a783685c88b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4232dfceee854234b309b18edb0e20d6'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:50.376745 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-84ffc3d7-a3f9-46bc-8927-8a783685c88b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4232dfceee854234b309b18edb0e20d6'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:50.404051 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-84ffc3d7-a3f9-46bc-8927-8a783685c88b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4232dfceee854234b309b18edb0e20d6'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:50.405627 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-84ffc3d7-a3f9-46bc-8927-8a783685c88b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4232dfceee854234b309b18edb0e20d6'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:50.487202 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1111/2229] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:32:49 2022] DELETE /identity/v3/users/4232dfceee854234b309b18edb0e20d6 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:51.038018 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a35967-5b6a-4ebb-b4e3-b203a093d913 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:51.053566 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a35967-5b6a-4ebb-b4e3-b203a093d913 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:51.148197 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:51.199456 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f849e1-2f43-4b8e-b246-2a8b446328f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:51.200171 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f849e1-2f43-4b8e-b246-2a8b446328f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:51.200465 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f849e1-2f43-4b8e-b246-2a8b446328f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:51.200751 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f849e1-2f43-4b8e-b246-2a8b446328f5 admin admin] PATH_INFO: `/v3/projects/ff67433a68ef49b0b243549a13e8bffa` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:51.202512 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f849e1-2f43-4b8e-b246-2a8b446328f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff67433a68ef49b0b243549a13e8bffa)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:51.205459 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f849e1-2f43-4b8e-b246-2a8b446328f5 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:51.373615 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-76f849e1-2f43-4b8e-b246-2a8b446328f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff67433a68ef49b0b243549a13e8bffa', 'request_id': 'req-76f849e1-2f43-4b8e-b246-2a8b446328f5'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:51.393862 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-76f849e1-2f43-4b8e-b246-2a8b446328f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff67433a68ef49b0b243549a13e8bffa', 'request_id': 'req-76f849e1-2f43-4b8e-b246-2a8b446328f5'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:51.405323 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-76f849e1-2f43-4b8e-b246-2a8b446328f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff67433a68ef49b0b243549a13e8bffa', 'request_id': 'req-76f849e1-2f43-4b8e-b246-2a8b446328f5'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:51.475434 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-76f849e1-2f43-4b8e-b246-2a8b446328f5 admin admin] The token cache is being invalidate because project ff67433a68ef49b0b243549a13e8bffa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:32:51.475823 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-76f849e1-2f43-4b8e-b246-2a8b446328f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:51.479342 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-76f849e1-2f43-4b8e-b246-2a8b446328f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:51.481197 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1119/2230] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:32:51 2022] DELETE /identity/v3/projects/ff67433a68ef49b0b243549a13e8bffa => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:51.486222 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ffc3d7-a3f9-46bc-8927-8a783685c88b admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:51.488831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-84ffc3d7-a3f9-46bc-8927-8a783685c88b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:51.524361 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ffc3d7-a3f9-46bc-8927-8a783685c88b admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:51.614982 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7942a2fa-6244-48c9-8efd-2b5430597a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:51.615689 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942a2fa-6244-48c9-8efd-2b5430597a49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:51.616256 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942a2fa-6244-48c9-8efd-2b5430597a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:51.617017 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942a2fa-6244-48c9-8efd-2b5430597a49 admin admin] PATH_INFO: `/v3/users/4c081d957fff49d98e259e14d054666f` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:51.618427 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7942a2fa-6244-48c9-8efd-2b5430597a49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c081d957fff49d98e259e14d054666f)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:51.620967 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7942a2fa-6244-48c9-8efd-2b5430597a49 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:51.894722 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-7942a2fa-6244-48c9-8efd-2b5430597a49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c081d957fff49d98e259e14d054666f'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:51.907315 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-7942a2fa-6244-48c9-8efd-2b5430597a49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c081d957fff49d98e259e14d054666f'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:51.917203 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-7942a2fa-6244-48c9-8efd-2b5430597a49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c081d957fff49d98e259e14d054666f'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:51.946715 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-7942a2fa-6244-48c9-8efd-2b5430597a49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c081d957fff49d98e259e14d054666f'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:51.947370 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-7942a2fa-6244-48c9-8efd-2b5430597a49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c081d957fff49d98e259e14d054666f'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:52.073141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1112/2231] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:32:51 2022] DELETE /identity/v3/users/4c081d957fff49d98e259e14d054666f => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:52.999191 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f849e1-2f43-4b8e-b246-2a8b446328f5 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:53.000895 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-76f849e1-2f43-4b8e-b246-2a8b446328f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:53.022209 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f849e1-2f43-4b8e-b246-2a8b446328f5 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:53.203221 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cdb225-fc47-4721-ade4-4e741972a189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:53.204839 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cdb225-fc47-4721-ade4-4e741972a189 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:53.205341 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cdb225-fc47-4721-ade4-4e741972a189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:53.206050 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cdb225-fc47-4721-ade4-4e741972a189 admin admin] PATH_INFO: `/v3/projects/f150318010e146b3a2d4390e2709d1d2` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:53.208362 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cdb225-fc47-4721-ade4-4e741972a189 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f150318010e146b3a2d4390e2709d1d2)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:53.210701 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cdb225-fc47-4721-ade4-4e741972a189 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:53.349929 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-d0cdb225-fc47-4721-ade4-4e741972a189 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f150318010e146b3a2d4390e2709d1d2', 'request_id': 'req-d0cdb225-fc47-4721-ade4-4e741972a189'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:53.361046 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-d0cdb225-fc47-4721-ade4-4e741972a189 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f150318010e146b3a2d4390e2709d1d2', 'request_id': 'req-d0cdb225-fc47-4721-ade4-4e741972a189'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:53.371685 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-d0cdb225-fc47-4721-ade4-4e741972a189 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f150318010e146b3a2d4390e2709d1d2', 'request_id': 'req-d0cdb225-fc47-4721-ade4-4e741972a189'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:53.468323 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:53.479013 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-d0cdb225-fc47-4721-ade4-4e741972a189 admin admin] The token cache is being invalidate because project f150318010e146b3a2d4390e2709d1d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:32:53.479013 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-d0cdb225-fc47-4721-ade4-4e741972a189 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:53.480593 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-d0cdb225-fc47-4721-ade4-4e741972a189 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:53.483995 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1120/2232] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:32:52 2022] DELETE /identity/v3/projects/f150318010e146b3a2d4390e2709d1d2 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:53.492754 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12fc7d5-c81f-4421-b58e-9bcdb5d0aa36 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:53.493309 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12fc7d5-c81f-4421-b58e-9bcdb5d0aa36 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:53.493309 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12fc7d5-c81f-4421-b58e-9bcdb5d0aa36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:53.554548 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-b12fc7d5-c81f-4421-b58e-9bcdb5d0aa36 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:53.556264 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-b12fc7d5-c81f-4421-b58e-9bcdb5d0aa36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:53.608951 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:53.674757 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1113/2233] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:53.686045 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43659b63-aa95-405e-9efd-de65ef8b0c9c None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:53.686045 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43659b63-aa95-405e-9efd-de65ef8b0c9c None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:53.687538 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43659b63-aa95-405e-9efd-de65ef8b0c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:53.740662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-43659b63-aa95-405e-9efd-de65ef8b0c9c None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:53.742016 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-43659b63-aa95-405e-9efd-de65ef8b0c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:53.755375 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1121/2234] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:53.759582 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12fc7d5-c81f-4421-b58e-9bcdb5d0aa36 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:53.771324 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12fc7d5-c81f-4421-b58e-9bcdb5d0aa36 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:53.777756 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c388501-bb09-47d5-a303-f3f0471f822d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:53.778513 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c388501-bb09-47d5-a303-f3f0471f822d admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:53.779290 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c388501-bb09-47d5-a303-f3f0471f822d admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:53.780511 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c388501-bb09-47d5-a303-f3f0471f822d admin admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:53.781414 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c388501-bb09-47d5-a303-f3f0471f822d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:32:53.782161 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c388501-bb09-47d5-a303-f3f0471f822d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:53.783999 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c388501-bb09-47d5-a303-f3f0471f822d admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:53.801374 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1114/2235] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:32:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:32:53.802596 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43659b63-aa95-405e-9efd-de65ef8b0c9c None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:53.819708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43659b63-aa95-405e-9efd-de65ef8b0c9c None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:53.820726 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a7299-7ca2-4e9c-b88e-872385dc3016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:53.821417 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a7299-7ca2-4e9c-b88e-872385dc3016 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:53.821918 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a7299-7ca2-4e9c-b88e-872385dc3016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:53.822278 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a7299-7ca2-4e9c-b88e-872385dc3016 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:53.823424 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892a7299-7ca2-4e9c-b88e-872385dc3016 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:53.825040 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892a7299-7ca2-4e9c-b88e-872385dc3016 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:53.966414 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1122/2236] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:32:53 2022] POST /identity/v3/projects => generated 381 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:32:53.970653 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c388501-bb09-47d5-a303-f3f0471f822d admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:53.984998 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c388501-bb09-47d5-a303-f3f0471f822d admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:54.124931 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb26e80-39f7-4d21-9c34-68b4eb2e1cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:54.125546 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb26e80-39f7-4d21-9c34-68b4eb2e1cfc admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:54.125832 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb26e80-39f7-4d21-9c34-68b4eb2e1cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:54.126112 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb26e80-39f7-4d21-9c34-68b4eb2e1cfc admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:54.126801 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb26e80-39f7-4d21-9c34-68b4eb2e1cfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:54.128815 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb26e80-39f7-4d21-9c34-68b4eb2e1cfc admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:54.253168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1115/2237] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:32:53 2022] POST /identity/v3/users => generated 291 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:32:54.261142 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a7299-7ca2-4e9c-b88e-872385dc3016 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:54.270209 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a7299-7ca2-4e9c-b88e-872385dc3016 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:54.275201 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f536c88b-0f74-4806-a00f-f8d5d07f3b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:54.275854 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536c88b-0f74-4806-a00f-f8d5d07f3b17 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:54.276739 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536c88b-0f74-4806-a00f-f8d5d07f3b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:54.277082 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536c88b-0f74-4806-a00f-f8d5d07f3b17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:54.277856 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f536c88b-0f74-4806-a00f-f8d5d07f3b17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:54.280697 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f536c88b-0f74-4806-a00f-f8d5d07f3b17 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:54.296160 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1123/2238] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:32:54.296880 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb26e80-39f7-4d21-9c34-68b4eb2e1cfc admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:54.313997 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb26e80-39f7-4d21-9c34-68b4eb2e1cfc admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:54.328985 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e12a2-7e8c-4fed-9bb5-1c56a1a5ac94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:54.329915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e12a2-7e8c-4fed-9bb5-1c56a1a5ac94 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:54.330391 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e12a2-7e8c-4fed-9bb5-1c56a1a5ac94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:54.330825 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e12a2-7e8c-4fed-9bb5-1c56a1a5ac94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:54.332261 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1e12a2-7e8c-4fed-9bb5-1c56a1a5ac94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:54.334536 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1e12a2-7e8c-4fed-9bb5-1c56a1a5ac94 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:54.353360 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1116/2239] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:54 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:32:54.357385 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f536c88b-0f74-4806-a00f-f8d5d07f3b17 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:54.370821 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f536c88b-0f74-4806-a00f-f8d5d07f3b17 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:54.377063 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf0fc11-33dd-4268-a6ff-2dd004ca7f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:54.377873 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf0fc11-33dd-4268-a6ff-2dd004ca7f3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:54.381338 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf0fc11-33dd-4268-a6ff-2dd004ca7f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:54.381338 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf0fc11-33dd-4268-a6ff-2dd004ca7f3e admin admin] PATH_INFO: `/v3/projects/818a938fda0f469c8f91bfcecf0512c2/users/a5d78489bbf74e479b5e827825327f4a/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:54.398454 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf0fc11-33dd-4268-a6ff-2dd004ca7f3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=818a938fda0f469c8f91bfcecf0512c2, user_id=a5d78489bbf74e479b5e827825327f4a, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:54.401815 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf0fc11-33dd-4268-a6ff-2dd004ca7f3e admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:54.536461 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1124/2240] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:32:54 2022] PUT /identity/v3/projects/818a938fda0f469c8f91bfcecf0512c2/users/a5d78489bbf74e479b5e827825327f4a/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:32:54.549929 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7dc6a9-a589-47b8-87a7-4fbac3cdb69d None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:54.551076 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7dc6a9-a589-47b8-87a7-4fbac3cdb69d None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:54.551529 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7dc6a9-a589-47b8-87a7-4fbac3cdb69d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:54.598965 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-db7dc6a9-a589-47b8-87a7-4fbac3cdb69d None None] MFA Rules not processed for user `a5d78489bbf74e479b5e827825327f4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:54.601190 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-db7dc6a9-a589-47b8-87a7-4fbac3cdb69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:54.699881 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf0fc11-33dd-4268-a6ff-2dd004ca7f3e admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:54.712169 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1117/2241] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:54.796573 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf0fc11-33dd-4268-a6ff-2dd004ca7f3e admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:54.803822 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556de6bf-11f0-4042-b6f0-60a9320a2ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:54.804608 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556de6bf-11f0-4042-b6f0-60a9320a2ab2 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:54.804897 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556de6bf-11f0-4042-b6f0-60a9320a2ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:54.805174 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556de6bf-11f0-4042-b6f0-60a9320a2ab2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:54.805872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556de6bf-11f0-4042-b6f0-60a9320a2ab2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:54.809692 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556de6bf-11f0-4042-b6f0-60a9320a2ab2 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:54.930104 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1125/2242] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:32:54 2022] POST /identity/v3/projects => generated 383 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:32:54.934198 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7dc6a9-a589-47b8-87a7-4fbac3cdb69d None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:54.950467 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7dc6a9-a589-47b8-87a7-4fbac3cdb69d None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:54.992580 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:55.043940 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdcbcae-915c-4f5b-8788-89ca87b266dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:55.044624 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdcbcae-915c-4f5b-8788-89ca87b266dd admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:55.045592 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdcbcae-915c-4f5b-8788-89ca87b266dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:55.045805 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdcbcae-915c-4f5b-8788-89ca87b266dd admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:55.047164 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdcbcae-915c-4f5b-8788-89ca87b266dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:55.049326 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdcbcae-915c-4f5b-8788-89ca87b266dd admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:55.129380 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556de6bf-11f0-4042-b6f0-60a9320a2ab2 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:55.129736 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1118/2243] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:32:54 2022] POST /identity/v3/users => generated 298 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:32:55.162534 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556de6bf-11f0-4042-b6f0-60a9320a2ab2 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:55.190713 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f6982d-6bd2-49f7-9c77-e6bbc6a6a711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:55.192103 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6982d-6bd2-49f7-9c77-e6bbc6a6a711 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:55.192604 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6982d-6bd2-49f7-9c77-e6bbc6a6a711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:55.193070 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6982d-6bd2-49f7-9c77-e6bbc6a6a711 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:55.194901 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f6982d-6bd2-49f7-9c77-e6bbc6a6a711 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:55.196766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f6982d-6bd2-49f7-9c77-e6bbc6a6a711 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:55.211880 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1126/2244] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:55 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:32:55.216696 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdcbcae-915c-4f5b-8788-89ca87b266dd admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:55.232492 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdcbcae-915c-4f5b-8788-89ca87b266dd admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:55.240160 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27a705-cfb4-4b9c-9cba-c78ac6fd3a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:55.240994 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27a705-cfb4-4b9c-9cba-c78ac6fd3a16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:55.241348 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27a705-cfb4-4b9c-9cba-c78ac6fd3a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:55.241698 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27a705-cfb4-4b9c-9cba-c78ac6fd3a16 admin admin] PATH_INFO: `/v3/projects/448b0684dc734adbb0716d950dae3ea6/users/5fe1cd9452fd4664930e7248a34616e1/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:55.270003 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b27a705-cfb4-4b9c-9cba-c78ac6fd3a16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=448b0684dc734adbb0716d950dae3ea6, user_id=5fe1cd9452fd4664930e7248a34616e1, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:55.274288 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b27a705-cfb4-4b9c-9cba-c78ac6fd3a16 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:55.375385 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1119/2245] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:32:55 2022] PUT /identity/v3/projects/448b0684dc734adbb0716d950dae3ea6/users/5fe1cd9452fd4664930e7248a34616e1/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:55.378734 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f6982d-6bd2-49f7-9c77-e6bbc6a6a711 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:55.394667 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f6982d-6bd2-49f7-9c77-e6bbc6a6a711 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:55.530284 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde19602-db06-4694-b081-3c98f31a8e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:55.530772 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde19602-db06-4694-b081-3c98f31a8e00 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:55.531205 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde19602-db06-4694-b081-3c98f31a8e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:55.532001 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde19602-db06-4694-b081-3c98f31a8e00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:55.533003 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde19602-db06-4694-b081-3c98f31a8e00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:55.535460 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde19602-db06-4694-b081-3c98f31a8e00 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:55.554073 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1127/2246] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:32:55 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:32:55.555147 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27a705-cfb4-4b9c-9cba-c78ac6fd3a16 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:55.571856 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27a705-cfb4-4b9c-9cba-c78ac6fd3a16 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:55.578951 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a97cac4-f570-4229-9bfa-50b8871336e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:55.579703 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97cac4-f570-4229-9bfa-50b8871336e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:55.579988 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97cac4-f570-4229-9bfa-50b8871336e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:55.580306 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97cac4-f570-4229-9bfa-50b8871336e6 admin admin] PATH_INFO: `/v3/projects/448b0684dc734adbb0716d950dae3ea6/users/5fe1cd9452fd4664930e7248a34616e1/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:55.582831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a97cac4-f570-4229-9bfa-50b8871336e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=448b0684dc734adbb0716d950dae3ea6, user_id=5fe1cd9452fd4664930e7248a34616e1, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:55.587169 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a97cac4-f570-4229-9bfa-50b8871336e6 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:55.609007 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1120/2247] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:32:55 2022] PUT /identity/v3/projects/448b0684dc734adbb0716d950dae3ea6/users/5fe1cd9452fd4664930e7248a34616e1/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:32:55.611364 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa917fe-062b-4348-b713-50a9c8148894 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:55.611836 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa917fe-062b-4348-b713-50a9c8148894 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:55.612145 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa917fe-062b-4348-b713-50a9c8148894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:55.635427 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:55.655190 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-7aa917fe-062b-4348-b713-50a9c8148894 None None] MFA Rules not processed for user `5fe1cd9452fd4664930e7248a34616e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:55.656553 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-7aa917fe-062b-4348-b713-50a9c8148894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:55.738029 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1128/2248] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:55 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:55.751986 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a97cac4-f570-4229-9bfa-50b8871336e6 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:55.754128 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-2a97cac4-f570-4229-9bfa-50b8871336e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:55.779814 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f644423-e5bf-4307-b4bc-7c4c38ccdaa4 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:55.780405 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f644423-e5bf-4307-b4bc-7c4c38ccdaa4 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:55.780510 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f644423-e5bf-4307-b4bc-7c4c38ccdaa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:55.825664 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-3f644423-e5bf-4307-b4bc-7c4c38ccdaa4 None None] MFA Rules not processed for user `cb055110c77543c1861dfd8c155d7582`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:55.829340 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-3f644423-e5bf-4307-b4bc-7c4c38ccdaa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:55.897218 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a97cac4-f570-4229-9bfa-50b8871336e6 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:55.906313 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae064a8-b5a0-4256-9920-fbcee609e32e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:55.909531 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae064a8-b5a0-4256-9920-fbcee609e32e service cinder] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:55.912251 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae064a8-b5a0-4256-9920-fbcee609e32e service cinder] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:55.912784 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae064a8-b5a0-4256-9920-fbcee609e32e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:55.942359 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1129/2249] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:55 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:32:55.945879 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae064a8-b5a0-4256-9920-fbcee609e32e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:55.949955 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae064a8-b5a0-4256-9920-fbcee609e32e service cinder] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:55.974896 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1121/2250] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:32:55 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:32:56.130379 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f644423-e5bf-4307-b4bc-7c4c38ccdaa4 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:56.143216 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f644423-e5bf-4307-b4bc-7c4c38ccdaa4 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:56.148163 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b120f7f-870b-41d8-9df7-371ebee9a903 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:56.148880 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b120f7f-870b-41d8-9df7-371ebee9a903 service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:56.149235 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b120f7f-870b-41d8-9df7-371ebee9a903 service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:56.149591 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b120f7f-870b-41d8-9df7-371ebee9a903 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:56.299370 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b120f7f-870b-41d8-9df7-371ebee9a903 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:56.304204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b120f7f-870b-41d8-9df7-371ebee9a903 service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:56.329090 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1130/2251] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:32:56 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:32:56.465423 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae064a8-b5a0-4256-9920-fbcee609e32e service cinder] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:56.468310 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-bae064a8-b5a0-4256-9920-fbcee609e32e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:56.570307 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae064a8-b5a0-4256-9920-fbcee609e32e service cinder] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:56.586574 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9053e821-36c4-4301-9c74-bf5ca6b9f0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:56.588176 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053e821-36c4-4301-9c74-bf5ca6b9f0b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:56.597502 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053e821-36c4-4301-9c74-bf5ca6b9f0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:56.597502 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053e821-36c4-4301-9c74-bf5ca6b9f0b1 admin admin] PATH_INFO: `/v3/users/5491ed66365140b9910a10d618a404f8` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:56.597502 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9053e821-36c4-4301-9c74-bf5ca6b9f0b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5491ed66365140b9910a10d618a404f8)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:56.600031 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9053e821-36c4-4301-9c74-bf5ca6b9f0b1 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:56.674072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:56.907310 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-9053e821-36c4-4301-9c74-bf5ca6b9f0b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5491ed66365140b9910a10d618a404f8'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:56.923222 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-9053e821-36c4-4301-9c74-bf5ca6b9f0b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5491ed66365140b9910a10d618a404f8'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:56.924053 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-9053e821-36c4-4301-9c74-bf5ca6b9f0b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5491ed66365140b9910a10d618a404f8'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:56.964136 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-9053e821-36c4-4301-9c74-bf5ca6b9f0b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5491ed66365140b9910a10d618a404f8'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:56.966353 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-9053e821-36c4-4301-9c74-bf5ca6b9f0b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5491ed66365140b9910a10d618a404f8'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:57.081728 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1122/2252] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:32:56 2022] DELETE /identity/v3/users/5491ed66365140b9910a10d618a404f8 => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:57.119125 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b120f7f-870b-41d8-9df7-371ebee9a903 service cinder] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:57.147240 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b120f7f-870b-41d8-9df7-371ebee9a903 service cinder] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:57.248933 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:57.284923 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1dd9d-c271-491f-91cb-cb8c0d4a794b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:57.285617 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1dd9d-c271-491f-91cb-cb8c0d4a794b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:57.285901 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1dd9d-c271-491f-91cb-cb8c0d4a794b admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:57.286180 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1dd9d-c271-491f-91cb-cb8c0d4a794b admin admin] PATH_INFO: `/v3/users/a5d78489bbf74e479b5e827825327f4a` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:57.287395 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d1dd9d-c271-491f-91cb-cb8c0d4a794b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5d78489bbf74e479b5e827825327f4a)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:57.289995 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d1dd9d-c271-491f-91cb-cb8c0d4a794b admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:57.554335 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-a3d1dd9d-c271-491f-91cb-cb8c0d4a794b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5d78489bbf74e479b5e827825327f4a'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:57.555093 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-a3d1dd9d-c271-491f-91cb-cb8c0d4a794b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5d78489bbf74e479b5e827825327f4a'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:57.610362 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5741e6-81b3-448a-91ca-214d733a315b None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:57.610837 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5741e6-81b3-448a-91ca-214d733a315b None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:57.611294 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5741e6-81b3-448a-91ca-214d733a315b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:57.646230 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-a3d1dd9d-c271-491f-91cb-cb8c0d4a794b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5d78489bbf74e479b5e827825327f4a'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:57.663529 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-6e5741e6-81b3-448a-91ca-214d733a315b None None] MFA Rules not processed for user `d62b8944fe534f10b71e305315a75f0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:57.665101 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-6e5741e6-81b3-448a-91ca-214d733a315b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:57.733770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-a3d1dd9d-c271-491f-91cb-cb8c0d4a794b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5d78489bbf74e479b5e827825327f4a'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:57.750373 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-a3d1dd9d-c271-491f-91cb-cb8c0d4a794b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5d78489bbf74e479b5e827825327f4a'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:57.763525 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1131/2253] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:32:57 2022] DELETE /identity/v3/users/a5d78489bbf74e479b5e827825327f4a => generated 0 bytes in 643 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:57.765104 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1dd9d-c271-491f-91cb-cb8c0d4a794b admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:57.766658 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-a3d1dd9d-c271-491f-91cb-cb8c0d4a794b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:57.804096 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1123/2254] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:57 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:57.804096 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2ba65-7dae-4a6b-aced-0eebfd6da633 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:57.804096 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2ba65-7dae-4a6b-aced-0eebfd6da633 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:57.804096 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2ba65-7dae-4a6b-aced-0eebfd6da633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:57.839546 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-e4a2ba65-7dae-4a6b-aced-0eebfd6da633 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:57.840925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-e4a2ba65-7dae-4a6b-aced-0eebfd6da633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:57.879873 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1dd9d-c271-491f-91cb-cb8c0d4a794b admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:57.885335 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7d8eda-51ed-4a0c-b72b-0341e502eef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:57.886330 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d8eda-51ed-4a0c-b72b-0341e502eef9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:57.886777 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d8eda-51ed-4a0c-b72b-0341e502eef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:57.887055 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d8eda-51ed-4a0c-b72b-0341e502eef9 admin admin] PATH_INFO: `/v3/projects/8a27b3e9643741e486fc931afee737bf` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:57.888394 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7d8eda-51ed-4a0c-b72b-0341e502eef9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a27b3e9643741e486fc931afee737bf)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:57.891797 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7d8eda-51ed-4a0c-b72b-0341e502eef9 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:57.937548 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1124/2255] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:32:57.947357 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a2ba65-7dae-4a6b-aced-0eebfd6da633 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:57.949480 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-e4a2ba65-7dae-4a6b-aced-0eebfd6da633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:57.998206 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:58.040364 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a2ba65-7dae-4a6b-aced-0eebfd6da633 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:58.044680 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a065ad2-532f-4e1c-8424-5b2622bdec92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:58.049053 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a065ad2-532f-4e1c-8424-5b2622bdec92 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:58.049053 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a065ad2-532f-4e1c-8424-5b2622bdec92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:58.049053 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a065ad2-532f-4e1c-8424-5b2622bdec92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:58.065551 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a065ad2-532f-4e1c-8424-5b2622bdec92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:58.069087 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a065ad2-532f-4e1c-8424-5b2622bdec92 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:58.069684 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-9f7d8eda-51ed-4a0c-b72b-0341e502eef9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a27b3e9643741e486fc931afee737bf', 'request_id': 'req-9f7d8eda-51ed-4a0c-b72b-0341e502eef9'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:58.088944 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1125/2256] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:32:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 15:32:58.088944 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a065ad2-532f-4e1c-8424-5b2622bdec92 service neutron] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:58.089631 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-9a065ad2-532f-4e1c-8424-5b2622bdec92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:58.093348 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-9f7d8eda-51ed-4a0c-b72b-0341e502eef9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a27b3e9643741e486fc931afee737bf', 'request_id': 'req-9f7d8eda-51ed-4a0c-b72b-0341e502eef9'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:58.097454 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:58.103076 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-9f7d8eda-51ed-4a0c-b72b-0341e502eef9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a27b3e9643741e486fc931afee737bf', 'request_id': 'req-9f7d8eda-51ed-4a0c-b72b-0341e502eef9'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:58.190979 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a065ad2-532f-4e1c-8424-5b2622bdec92 service neutron] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:58.253759 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-9f7d8eda-51ed-4a0c-b72b-0341e502eef9 admin admin] The token cache is being invalidate because project 8a27b3e9643741e486fc931afee737bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:32:58.255584 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-9f7d8eda-51ed-4a0c-b72b-0341e502eef9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:58.256894 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-9f7d8eda-51ed-4a0c-b72b-0341e502eef9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:58.258886 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1132/2257] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:32:57 2022] DELETE /identity/v3/projects/8a27b3e9643741e486fc931afee737bf => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:58.263217 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7d8eda-51ed-4a0c-b72b-0341e502eef9 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:58.265690 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-9f7d8eda-51ed-4a0c-b72b-0341e502eef9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:58.348187 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9248f549-9d31-4b4d-8f0b-4a69e76b4190 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:58.350470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9248f549-9d31-4b4d-8f0b-4a69e76b4190 service nova] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:58.350917 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9248f549-9d31-4b4d-8f0b-4a69e76b4190 service nova] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:58.352297 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9248f549-9d31-4b4d-8f0b-4a69e76b4190 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:58.355645 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-9248f549-9d31-4b4d-8f0b-4a69e76b4190 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:58.442121 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7d8eda-51ed-4a0c-b72b-0341e502eef9 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:58.449626 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e904fec-0890-48c6-8f68-ffe380a8e298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:58.450594 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e904fec-0890-48c6-8f68-ffe380a8e298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:58.451103 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e904fec-0890-48c6-8f68-ffe380a8e298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:58.453048 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e904fec-0890-48c6-8f68-ffe380a8e298 admin admin] PATH_INFO: `/v3/users/24ed074740414ce8868720f7f55e6cda` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:58.453835 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e904fec-0890-48c6-8f68-ffe380a8e298 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24ed074740414ce8868720f7f55e6cda)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:58.455746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e904fec-0890-48c6-8f68-ffe380a8e298 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:58.468263 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9248f549-9d31-4b4d-8f0b-4a69e76b4190 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:58.470042 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9248f549-9d31-4b4d-8f0b-4a69e76b4190 service nova] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:58.484667 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1126/2258] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:32:58 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:32:58.668550 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-7e904fec-0890-48c6-8f68-ffe380a8e298 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24ed074740414ce8868720f7f55e6cda'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:58.669067 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-7e904fec-0890-48c6-8f68-ffe380a8e298 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24ed074740414ce8868720f7f55e6cda'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:58.687054 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-7e904fec-0890-48c6-8f68-ffe380a8e298 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24ed074740414ce8868720f7f55e6cda'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:58.783277 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-7e904fec-0890-48c6-8f68-ffe380a8e298 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24ed074740414ce8868720f7f55e6cda'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:58.801837 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-7e904fec-0890-48c6-8f68-ffe380a8e298 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24ed074740414ce8868720f7f55e6cda'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:58.809021 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1133/2259] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:32:58 2022] DELETE /identity/v3/users/24ed074740414ce8868720f7f55e6cda => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:58.969299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9248f549-9d31-4b4d-8f0b-4a69e76b4190 service nova] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:58.974066 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-9248f549-9d31-4b4d-8f0b-4a69e76b4190 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:58.987786 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:58.997545 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9248f549-9d31-4b4d-8f0b-4a69e76b4190 service nova] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:59.090157 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c4124-134e-498f-9824-d0faf5c335b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:59.090821 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c4124-134e-498f-9824-d0faf5c335b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:59.091109 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c4124-134e-498f-9824-d0faf5c335b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:59.091389 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c4124-134e-498f-9824-d0faf5c335b7 admin admin] PATH_INFO: `/v3/projects/818a938fda0f469c8f91bfcecf0512c2` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:59.093531 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e904fec-0890-48c6-8f68-ffe380a8e298 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:59.095593 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-7e904fec-0890-48c6-8f68-ffe380a8e298 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:59.106101 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c4124-134e-498f-9824-d0faf5c335b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=818a938fda0f469c8f91bfcecf0512c2)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:59.113538 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c4124-134e-498f-9824-d0faf5c335b7 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:59.118594 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e904fec-0890-48c6-8f68-ffe380a8e298 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:59.135801 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92d4eea-6093-45c1-b911-f1f40d93f918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:59.136515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d4eea-6093-45c1-b911-f1f40d93f918 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:59.136810 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d4eea-6093-45c1-b911-f1f40d93f918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:59.137137 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d4eea-6093-45c1-b911-f1f40d93f918 admin admin] PATH_INFO: `/v3/projects/190e2e9877414b9eb6bf1f141ea14711` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:59.139473 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92d4eea-6093-45c1-b911-f1f40d93f918 admin admin] RBAC: Authorizing `identity:delete_project(project_id=190e2e9877414b9eb6bf1f141ea14711)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:59.141469 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92d4eea-6093-45c1-b911-f1f40d93f918 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:59.301930 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-9b4c4124-134e-498f-9824-d0faf5c335b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '818a938fda0f469c8f91bfcecf0512c2', 'request_id': 'req-9b4c4124-134e-498f-9824-d0faf5c335b7'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:59.314178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-9b4c4124-134e-498f-9824-d0faf5c335b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '818a938fda0f469c8f91bfcecf0512c2', 'request_id': 'req-9b4c4124-134e-498f-9824-d0faf5c335b7'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:59.343526 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c92d4eea-6093-45c1-b911-f1f40d93f918 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '190e2e9877414b9eb6bf1f141ea14711', 'request_id': 'req-c92d4eea-6093-45c1-b911-f1f40d93f918'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:59.350396 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c92d4eea-6093-45c1-b911-f1f40d93f918 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '190e2e9877414b9eb6bf1f141ea14711', 'request_id': 'req-c92d4eea-6093-45c1-b911-f1f40d93f918'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:59.356671 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:59.358643 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c92d4eea-6093-45c1-b911-f1f40d93f918 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '190e2e9877414b9eb6bf1f141ea14711', 'request_id': 'req-c92d4eea-6093-45c1-b911-f1f40d93f918'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:59.383873 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-9b4c4124-134e-498f-9824-d0faf5c335b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '818a938fda0f469c8f91bfcecf0512c2', 'request_id': 'req-9b4c4124-134e-498f-9824-d0faf5c335b7'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:59.425644 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-9b4c4124-134e-498f-9824-d0faf5c335b7 admin admin] The token cache is being invalidate because project 818a938fda0f469c8f91bfcecf0512c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:32:59.425936 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-9b4c4124-134e-498f-9824-d0faf5c335b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:59.432397 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-9b4c4124-134e-498f-9824-d0faf5c335b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:59.434413 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1127/2260] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:32:58 2022] DELETE /identity/v3/projects/818a938fda0f469c8f91bfcecf0512c2 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:59.439901 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c4124-134e-498f-9824-d0faf5c335b7 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:59.447985 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-9b4c4124-134e-498f-9824-d0faf5c335b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:59.509884 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c92d4eea-6093-45c1-b911-f1f40d93f918 admin admin] The token cache is being invalidate because project 190e2e9877414b9eb6bf1f141ea14711 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:32:59.510328 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c92d4eea-6093-45c1-b911-f1f40d93f918 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:59.511087 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c92d4eea-6093-45c1-b911-f1f40d93f918 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:32:59.514194 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1134/2261] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:32:59 2022] DELETE /identity/v3/projects/190e2e9877414b9eb6bf1f141ea14711 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:32:59.518726 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97de51b-38d9-4f7d-bb28-b4e4f73653ba None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:59.519259 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97de51b-38d9-4f7d-bb28-b4e4f73653ba None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:59.519782 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97de51b-38d9-4f7d-bb28-b4e4f73653ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:59.561644 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-b97de51b-38d9-4f7d-bb28-b4e4f73653ba None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:59.563019 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-b97de51b-38d9-4f7d-bb28-b4e4f73653ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:59.590306 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c4124-134e-498f-9824-d0faf5c335b7 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:59.641817 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:32:59.675661 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b098c0-0d94-4aa7-ac32-19df9c7bb23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:59.676582 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b098c0-0d94-4aa7-ac32-19df9c7bb23e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:59.677187 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b098c0-0d94-4aa7-ac32-19df9c7bb23e admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:59.677536 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b098c0-0d94-4aa7-ac32-19df9c7bb23e admin admin] PATH_INFO: `/v3/users/5fe1cd9452fd4664930e7248a34616e1` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:59.679196 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b098c0-0d94-4aa7-ac32-19df9c7bb23e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fe1cd9452fd4664930e7248a34616e1)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:59.682772 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b098c0-0d94-4aa7-ac32-19df9c7bb23e admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:59.691253 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1135/2262] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:32:59.699499 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8c664-733f-4bee-9d99-1c49413a7607 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:59.699976 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8c664-733f-4bee-9d99-1c49413a7607 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:59.700459 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8c664-733f-4bee-9d99-1c49413a7607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:59.876735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-8ac8c664-733f-4bee-9d99-1c49413a7607 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:32:59.876735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-8ac8c664-733f-4bee-9d99-1c49413a7607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:32:59.876735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1136/2263] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:32:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:32:59.893085 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8c664-733f-4bee-9d99-1c49413a7607 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:59.902912 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8c664-733f-4bee-9d99-1c49413a7607 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:59.908169 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62a86d-662e-4be2-958c-543f8b0f9b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:59.908917 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62a86d-662e-4be2-958c-543f8b0f9b5c admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:59.909207 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62a86d-662e-4be2-958c-543f8b0f9b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:59.909483 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62a86d-662e-4be2-958c-543f8b0f9b5c admin admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:59.910192 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc62a86d-662e-4be2-958c-543f8b0f9b5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:32:59.910500 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc62a86d-662e-4be2-958c-543f8b0f9b5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:59.912796 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc62a86d-662e-4be2-958c-543f8b0f9b5c admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:32:59.934180 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1137/2264] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:32:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:32:59.935327 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62a86d-662e-4be2-958c-543f8b0f9b5c admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:32:59.945844 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62a86d-662e-4be2-958c-543f8b0f9b5c admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:32:59.950578 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb7523-530a-47d3-af7c-00fd5ba049b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:32:59.951217 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb7523-530a-47d3-af7c-00fd5ba049b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:32:59.951506 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb7523-530a-47d3-af7c-00fd5ba049b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:32:59.951790 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb7523-530a-47d3-af7c-00fd5ba049b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:32:59.952496 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb7523-530a-47d3-af7c-00fd5ba049b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:32:59.954451 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb7523-530a-47d3-af7c-00fd5ba049b9 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:00.116831 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1138/2265] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:32:59 2022] POST /identity/v3/projects => generated 385 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:33:00.120831 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb7523-530a-47d3-af7c-00fd5ba049b9 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:00.195658 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb7523-530a-47d3-af7c-00fd5ba049b9 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:00.200975 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b62856f-76ac-4091-9871-e382d5e0d4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:00.203930 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b62856f-76ac-4091-9871-e382d5e0d4fa admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:00.203930 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b62856f-76ac-4091-9871-e382d5e0d4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:00.203930 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b62856f-76ac-4091-9871-e382d5e0d4fa admin admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:00.203930 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b62856f-76ac-4091-9871-e382d5e0d4fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:00.205996 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b62856f-76ac-4091-9871-e382d5e0d4fa admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:00.219895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-41b098c0-0d94-4aa7-ac32-19df9c7bb23e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fe1cd9452fd4664930e7248a34616e1'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:00.232797 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-41b098c0-0d94-4aa7-ac32-19df9c7bb23e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fe1cd9452fd4664930e7248a34616e1'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:00.233838 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-41b098c0-0d94-4aa7-ac32-19df9c7bb23e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fe1cd9452fd4664930e7248a34616e1'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:00.250971 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-41b098c0-0d94-4aa7-ac32-19df9c7bb23e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fe1cd9452fd4664930e7248a34616e1'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:00.252089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-41b098c0-0d94-4aa7-ac32-19df9c7bb23e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fe1cd9452fd4664930e7248a34616e1'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:00.343753 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1139/2266] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:33:00 2022] POST /identity/v3/users => generated 293 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:33:00.347740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b62856f-76ac-4091-9871-e382d5e0d4fa admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:00.353739 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-6b62856f-76ac-4091-9871-e382d5e0d4fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:00.359224 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1128/2267] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:32:59 2022] DELETE /identity/v3/users/5fe1cd9452fd4664930e7248a34616e1 => generated 0 bytes in 920 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:33:00.372581 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b62856f-76ac-4091-9871-e382d5e0d4fa admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:00.411600 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:33:00.439414 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63accb-f8da-43c4-9b49-262ba919be14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:00.440646 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63accb-f8da-43c4-9b49-262ba919be14 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:00.440943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63accb-f8da-43c4-9b49-262ba919be14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:00.441226 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63accb-f8da-43c4-9b49-262ba919be14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:00.441925 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63accb-f8da-43c4-9b49-262ba919be14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:00.443840 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63accb-f8da-43c4-9b49-262ba919be14 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:00.459019 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1140/2268] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:33:00 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 15:33:00.463118 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b098c0-0d94-4aa7-ac32-19df9c7bb23e admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:00.484808 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b098c0-0d94-4aa7-ac32-19df9c7bb23e admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:00.491741 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5b412a-acef-4614-a70c-35d0ae1ef1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:00.493104 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b412a-acef-4614-a70c-35d0ae1ef1f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:00.493535 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b412a-acef-4614-a70c-35d0ae1ef1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:00.494235 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b412a-acef-4614-a70c-35d0ae1ef1f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:00.495209 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5b412a-acef-4614-a70c-35d0ae1ef1f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:00.497311 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5b412a-acef-4614-a70c-35d0ae1ef1f6 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:00.524745 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1129/2269] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:33:00 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:33:00.526881 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63accb-f8da-43c4-9b49-262ba919be14 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:00.543272 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63accb-f8da-43c4-9b49-262ba919be14 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:00.556278 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570a3aa2-6862-42cd-a5ab-59cf4e364b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:00.557080 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a3aa2-6862-42cd-a5ab-59cf4e364b46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:00.557693 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a3aa2-6862-42cd-a5ab-59cf4e364b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:00.557826 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a3aa2-6862-42cd-a5ab-59cf4e364b46 admin admin] PATH_INFO: `/v3/projects/f8499e82fe374b429e8f1196da41bc9e/users/bfeaa73b06a944d38ca072608d7b5d1c/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:00.600601 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570a3aa2-6862-42cd-a5ab-59cf4e364b46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8499e82fe374b429e8f1196da41bc9e, user_id=bfeaa73b06a944d38ca072608d7b5d1c, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:00.608582 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570a3aa2-6862-42cd-a5ab-59cf4e364b46 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:00.736302 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1141/2270] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:33:00 2022] PUT /identity/v3/projects/f8499e82fe374b429e8f1196da41bc9e/users/bfeaa73b06a944d38ca072608d7b5d1c/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:33:00.745835 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7e485-9d52-46e3-af3e-a19c66ca57cb None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:00.748218 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7e485-9d52-46e3-af3e-a19c66ca57cb None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:00.748218 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7e485-9d52-46e3-af3e-a19c66ca57cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:00.785300 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-dcf7e485-9d52-46e3-af3e-a19c66ca57cb None None] MFA Rules not processed for user `bfeaa73b06a944d38ca072608d7b5d1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:33:00.786644 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-dcf7e485-9d52-46e3-af3e-a19c66ca57cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:00.939839 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1130/2271] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:33:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:33:00.952652 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570a3aa2-6862-42cd-a5ab-59cf4e364b46 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:00.954634 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-570a3aa2-6862-42cd-a5ab-59cf4e364b46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:01.117067 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570a3aa2-6862-42cd-a5ab-59cf4e364b46 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:01.130189 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a41d5-cfd7-4789-ab30-dd0f92c10422 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:01.135036 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a41d5-cfd7-4789-ab30-dd0f92c10422 service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:01.135328 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a41d5-cfd7-4789-ab30-dd0f92c10422 service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:01.135612 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a41d5-cfd7-4789-ab30-dd0f92c10422 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:01.149437 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91a41d5-cfd7-4789-ab30-dd0f92c10422 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:01.150855 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91a41d5-cfd7-4789-ab30-dd0f92c10422 service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:01.186680 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1142/2272] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:33:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:33:01.217433 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf7e485-9d52-46e3-af3e-a19c66ca57cb None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:01.282589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:33:01.331283 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf7e485-9d52-46e3-af3e-a19c66ca57cb None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:01.336585 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a41d5-cfd7-4789-ab30-dd0f92c10422 service cinder] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:01.338602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-a91a41d5-cfd7-4789-ab30-dd0f92c10422 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:01.348271 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856bb5c4-248d-4616-a94d-0b1fbb433130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:01.348950 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bb5c4-248d-4616-a94d-0b1fbb433130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:01.349242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bb5c4-248d-4616-a94d-0b1fbb433130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:01.349521 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bb5c4-248d-4616-a94d-0b1fbb433130 admin admin] PATH_INFO: `/v3/users/bfeaa73b06a944d38ca072608d7b5d1c` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:01.350882 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856bb5c4-248d-4616-a94d-0b1fbb433130 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfeaa73b06a944d38ca072608d7b5d1c)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:01.352883 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856bb5c4-248d-4616-a94d-0b1fbb433130 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:01.371117 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a41d5-cfd7-4789-ab30-dd0f92c10422 service cinder] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:01.388440 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf0928-8975-46f8-89ca-fe024a9fd287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:01.388440 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf0928-8975-46f8-89ca-fe024a9fd287 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:01.388440 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf0928-8975-46f8-89ca-fe024a9fd287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:01.388440 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf0928-8975-46f8-89ca-fe024a9fd287 admin admin] PATH_INFO: `/v3/projects/448b0684dc734adbb0716d950dae3ea6` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:01.389358 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcf0928-8975-46f8-89ca-fe024a9fd287 admin admin] RBAC: Authorizing `identity:delete_project(project_id=448b0684dc734adbb0716d950dae3ea6)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:01.390869 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcf0928-8975-46f8-89ca-fe024a9fd287 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:01.598587 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-3bcf0928-8975-46f8-89ca-fe024a9fd287 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '448b0684dc734adbb0716d950dae3ea6', 'request_id': 'req-3bcf0928-8975-46f8-89ca-fe024a9fd287'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:01.626523 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-3bcf0928-8975-46f8-89ca-fe024a9fd287 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '448b0684dc734adbb0716d950dae3ea6', 'request_id': 'req-3bcf0928-8975-46f8-89ca-fe024a9fd287'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:01.637156 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-3bcf0928-8975-46f8-89ca-fe024a9fd287 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '448b0684dc734adbb0716d950dae3ea6', 'request_id': 'req-3bcf0928-8975-46f8-89ca-fe024a9fd287'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:01.637985 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-856bb5c4-248d-4616-a94d-0b1fbb433130 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfeaa73b06a944d38ca072608d7b5d1c'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:01.659213 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-856bb5c4-248d-4616-a94d-0b1fbb433130 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfeaa73b06a944d38ca072608d7b5d1c'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:01.663127 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-856bb5c4-248d-4616-a94d-0b1fbb433130 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfeaa73b06a944d38ca072608d7b5d1c'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:01.683053 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-856bb5c4-248d-4616-a94d-0b1fbb433130 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfeaa73b06a944d38ca072608d7b5d1c'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:01.684185 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-856bb5c4-248d-4616-a94d-0b1fbb433130 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfeaa73b06a944d38ca072608d7b5d1c'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:01.726660 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-3bcf0928-8975-46f8-89ca-fe024a9fd287 admin admin] The token cache is being invalidate because project 448b0684dc734adbb0716d950dae3ea6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:33:01.727016 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-3bcf0928-8975-46f8-89ca-fe024a9fd287 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:01.728288 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-3bcf0928-8975-46f8-89ca-fe024a9fd287 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:01.734736 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1143/2273] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:33:01 2022] DELETE /identity/v3/projects/448b0684dc734adbb0716d950dae3ea6 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:33:01.736977 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e75286-7829-4893-9f1e-2ec8d6c1e1bb None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:01.737270 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e75286-7829-4893-9f1e-2ec8d6c1e1bb None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:01.737555 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e75286-7829-4893-9f1e-2ec8d6c1e1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:01.799403 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-b1e75286-7829-4893-9f1e-2ec8d6c1e1bb None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:33:01.800698 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-b1e75286-7829-4893-9f1e-2ec8d6c1e1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:01.828284 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1131/2274] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:33:01 2022] DELETE /identity/v3/users/bfeaa73b06a944d38ca072608d7b5d1c => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:33:01.834760 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588ccc1-afdc-4e0a-b1b8-b8bc6af4a2fb None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:01.835802 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588ccc1-afdc-4e0a-b1b8-b8bc6af4a2fb None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:01.835802 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588ccc1-afdc-4e0a-b1b8-b8bc6af4a2fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:01.881034 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-9588ccc1-afdc-4e0a-b1b8-b8bc6af4a2fb None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:33:01.881034 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-9588ccc1-afdc-4e0a-b1b8-b8bc6af4a2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:01.898778 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:33:01.957450 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1144/2275] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:33:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:33:01.965165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6593e-9dcb-42d7-9135-1ea219b99f7c None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:01.966304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6593e-9dcb-42d7-9135-1ea219b99f7c None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:01.966304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6593e-9dcb-42d7-9135-1ea219b99f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:02.003932 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-65c6593e-9dcb-42d7-9135-1ea219b99f7c None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:33:02.008342 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-65c6593e-9dcb-42d7-9135-1ea219b99f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:02.020606 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1132/2276] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:33:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:33:02.033011 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9588ccc1-afdc-4e0a-b1b8-b8bc6af4a2fb None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:02.034591 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-9588ccc1-afdc-4e0a-b1b8-b8bc6af4a2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:02.081084 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1145/2277] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:33:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:33:02.085681 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c6593e-9dcb-42d7-9135-1ea219b99f7c None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:02.099231 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c6593e-9dcb-42d7-9135-1ea219b99f7c None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:02.101234 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9588ccc1-afdc-4e0a-b1b8-b8bc6af4a2fb None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:02.103526 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a9cf1d-3db8-411e-a9fe-c2320b7f0863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:02.104196 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9cf1d-3db8-411e-a9fe-c2320b7f0863 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:02.104489 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9cf1d-3db8-411e-a9fe-c2320b7f0863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:02.104773 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9cf1d-3db8-411e-a9fe-c2320b7f0863 admin admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:02.105484 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a9cf1d-3db8-411e-a9fe-c2320b7f0863 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:33:02.105796 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a9cf1d-3db8-411e-a9fe-c2320b7f0863 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:02.106955 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ffa3c-1029-45c8-9093-8ff97dd20c10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:02.108125 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a9cf1d-3db8-411e-a9fe-c2320b7f0863 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:02.108783 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ffa3c-1029-45c8-9093-8ff97dd20c10 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:02.109403 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ffa3c-1029-45c8-9093-8ff97dd20c10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:02.110026 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ffa3c-1029-45c8-9093-8ff97dd20c10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:02.120743 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1146/2278] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:33:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:33:02.123195 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8ffa3c-1029-45c8-9093-8ff97dd20c10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:02.124744 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8ffa3c-1029-45c8-9093-8ff97dd20c10 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:02.124878 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a9cf1d-3db8-411e-a9fe-c2320b7f0863 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:02.135908 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a9cf1d-3db8-411e-a9fe-c2320b7f0863 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:02.136785 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1133/2279] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:33:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:33:02.142144 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be051c2-af38-47c2-ba66-1e9dccfef283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:02.142941 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be051c2-af38-47c2-ba66-1e9dccfef283 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:02.143126 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be051c2-af38-47c2-ba66-1e9dccfef283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:02.143403 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be051c2-af38-47c2-ba66-1e9dccfef283 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:02.144109 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be051c2-af38-47c2-ba66-1e9dccfef283 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:02.146993 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be051c2-af38-47c2-ba66-1e9dccfef283 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:02.342122 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1147/2280] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:33:02 2022] POST /identity/v3/projects => generated 383 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:33:02.357997 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ffa3c-1029-45c8-9093-8ff97dd20c10 service neutron] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:02.369939 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ffa3c-1029-45c8-9093-8ff97dd20c10 service neutron] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:02.451782 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aad38f-6556-4959-a5ba-2b20fd370577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:02.452847 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aad38f-6556-4959-a5ba-2b20fd370577 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:02.453247 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aad38f-6556-4959-a5ba-2b20fd370577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:02.453728 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aad38f-6556-4959-a5ba-2b20fd370577 admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:02.454924 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aad38f-6556-4959-a5ba-2b20fd370577 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:02.459346 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aad38f-6556-4959-a5ba-2b20fd370577 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:02.598008 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1134/2281] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:33:02 2022] POST /identity/v3/users => generated 292 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:33:02.601405 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be051c2-af38-47c2-ba66-1e9dccfef283 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:02.623647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be051c2-af38-47c2-ba66-1e9dccfef283 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:02.638350 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7578e9-f12d-4175-ab9e-a5e339ceda24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:02.639587 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7578e9-f12d-4175-ab9e-a5e339ceda24 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:02.639879 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7578e9-f12d-4175-ab9e-a5e339ceda24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:02.640180 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7578e9-f12d-4175-ab9e-a5e339ceda24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:02.640906 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7578e9-f12d-4175-ab9e-a5e339ceda24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:02.642562 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7578e9-f12d-4175-ab9e-a5e339ceda24 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:02.658671 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1148/2282] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:33:02.659983 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aad38f-6556-4959-a5ba-2b20fd370577 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:02.683013 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aad38f-6556-4959-a5ba-2b20fd370577 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:02.689867 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d1f738-a113-4ced-a348-553a4b412fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:02.690873 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1f738-a113-4ced-a348-553a4b412fd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:02.691436 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1f738-a113-4ced-a348-553a4b412fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:02.691968 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1f738-a113-4ced-a348-553a4b412fd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:02.698307 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d1f738-a113-4ced-a348-553a4b412fd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:02.700533 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d1f738-a113-4ced-a348-553a4b412fd6 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:02.718796 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1135/2283] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 15:33:02.723088 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7578e9-f12d-4175-ab9e-a5e339ceda24 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:02.735791 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7578e9-f12d-4175-ab9e-a5e339ceda24 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:02.742490 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67cb6fd-e3a2-441a-aaef-39de0e05fec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:02.743519 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67cb6fd-e3a2-441a-aaef-39de0e05fec7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:02.743815 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67cb6fd-e3a2-441a-aaef-39de0e05fec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:02.744164 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67cb6fd-e3a2-441a-aaef-39de0e05fec7 admin admin] PATH_INFO: `/v3/projects/80924db5df73440d8d549a70ded8c4b1/users/428e07a1a2214d129242ba76d7dc852e/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:02.762607 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67cb6fd-e3a2-441a-aaef-39de0e05fec7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80924db5df73440d8d549a70ded8c4b1, user_id=428e07a1a2214d129242ba76d7dc852e, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:02.765533 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67cb6fd-e3a2-441a-aaef-39de0e05fec7 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:02.859605 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d1f738-a113-4ced-a348-553a4b412fd6 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:02.862482 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-32d1f738-a113-4ced-a348-553a4b412fd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:02.885016 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1149/2284] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:33:02 2022] PUT /identity/v3/projects/80924db5df73440d8d549a70ded8c4b1/users/428e07a1a2214d129242ba76d7dc852e/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:33:02.885132 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d1f738-a113-4ced-a348-553a4b412fd6 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:02.894238 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b644b6-8167-4cc7-a908-d7103988d3b6 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:02.894532 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b644b6-8167-4cc7-a908-d7103988d3b6 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:02.894817 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b644b6-8167-4cc7-a908-d7103988d3b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:02.937083 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-50b644b6-8167-4cc7-a908-d7103988d3b6 None None] MFA Rules not processed for user `428e07a1a2214d129242ba76d7dc852e`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:33:02.939149 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-50b644b6-8167-4cc7-a908-d7103988d3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:03.017281 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a21ed-39ca-448d-ba60-a99a5c7bb53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:03.017281 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a21ed-39ca-448d-ba60-a99a5c7bb53e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:03.017281 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a21ed-39ca-448d-ba60-a99a5c7bb53e admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:03.017281 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a21ed-39ca-448d-ba60-a99a5c7bb53e admin admin] PATH_INFO: `/v3/projects/f8499e82fe374b429e8f1196da41bc9e` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:03.019185 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60a21ed-39ca-448d-ba60-a99a5c7bb53e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8499e82fe374b429e8f1196da41bc9e)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:03.021161 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60a21ed-39ca-448d-ba60-a99a5c7bb53e admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:03.090101 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1150/2285] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:33:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:33:03.102609 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b644b6-8167-4cc7-a908-d7103988d3b6 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:03.104562 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-50b644b6-8167-4cc7-a908-d7103988d3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:03.175678 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-f60a21ed-39ca-448d-ba60-a99a5c7bb53e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8499e82fe374b429e8f1196da41bc9e', 'request_id': 'req-f60a21ed-39ca-448d-ba60-a99a5c7bb53e'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:03.191764 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-f60a21ed-39ca-448d-ba60-a99a5c7bb53e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8499e82fe374b429e8f1196da41bc9e', 'request_id': 'req-f60a21ed-39ca-448d-ba60-a99a5c7bb53e'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:03.236059 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b644b6-8167-4cc7-a908-d7103988d3b6 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:03.286253 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-f60a21ed-39ca-448d-ba60-a99a5c7bb53e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8499e82fe374b429e8f1196da41bc9e', 'request_id': 'req-f60a21ed-39ca-448d-ba60-a99a5c7bb53e'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:03.304739 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-f60a21ed-39ca-448d-ba60-a99a5c7bb53e admin admin] The token cache is being invalidate because project f8499e82fe374b429e8f1196da41bc9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:33:03.305206 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-f60a21ed-39ca-448d-ba60-a99a5c7bb53e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:03.306485 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-f60a21ed-39ca-448d-ba60-a99a5c7bb53e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:03.313988 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1136/2286] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:33:02 2022] DELETE /identity/v3/projects/f8499e82fe374b429e8f1196da41bc9e => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:33:03.317747 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158aa220-ac39-420b-8bf0-3b0b92eea5d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:03.318590 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158aa220-ac39-420b-8bf0-3b0b92eea5d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:03.318956 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158aa220-ac39-420b-8bf0-3b0b92eea5d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:03.319324 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158aa220-ac39-420b-8bf0-3b0b92eea5d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:03.321472 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-158aa220-ac39-420b-8bf0-3b0b92eea5d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:03.322080 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5cf0e-a1aa-4d7d-be9f-f267527f7029 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:03.322727 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5cf0e-a1aa-4d7d-be9f-f267527f7029 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:03.323405 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5cf0e-a1aa-4d7d-be9f-f267527f7029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:03.351469 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-35e5cf0e-a1aa-4d7d-be9f-f267527f7029 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:33:03.357044 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-35e5cf0e-a1aa-4d7d-be9f-f267527f7029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:03.384847 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:33:03.409736 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158aa220-ac39-420b-8bf0-3b0b92eea5d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:03.415472 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158aa220-ac39-420b-8bf0-3b0b92eea5d2 service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:03.422777 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1151/2287] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:33:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 15:33:03.487621 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1137/2288] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:33:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:33:03.496025 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a514fdd-7ec3-4d30-9d5b-930a7b9af681 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:03.496589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a514fdd-7ec3-4d30-9d5b-930a7b9af681 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:03.496911 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a514fdd-7ec3-4d30-9d5b-930a7b9af681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:03.541284 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-9a514fdd-7ec3-4d30-9d5b-930a7b9af681 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:33:03.543342 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-9a514fdd-7ec3-4d30-9d5b-930a7b9af681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:03.558688 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1152/2289] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:33:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:33:03.563775 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5cf0e-a1aa-4d7d-be9f-f267527f7029 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:03.578114 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5cf0e-a1aa-4d7d-be9f-f267527f7029 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:03.585381 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829d90cb-09fc-48ee-891b-e3fbaa5f193e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:03.586012 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d90cb-09fc-48ee-891b-e3fbaa5f193e admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:03.586825 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d90cb-09fc-48ee-891b-e3fbaa5f193e admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:03.587112 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d90cb-09fc-48ee-891b-e3fbaa5f193e admin admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:03.587853 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829d90cb-09fc-48ee-891b-e3fbaa5f193e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:33:03.588196 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829d90cb-09fc-48ee-891b-e3fbaa5f193e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:03.589791 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829d90cb-09fc-48ee-891b-e3fbaa5f193e admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:03.607859 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1138/2290] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:33:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:33:03.609003 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a514fdd-7ec3-4d30-9d5b-930a7b9af681 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:03.618875 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a514fdd-7ec3-4d30-9d5b-930a7b9af681 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:03.625420 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f74f6-dcbe-41ec-8f4b-52929ba1e348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:03.625900 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f74f6-dcbe-41ec-8f4b-52929ba1e348 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:03.626187 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f74f6-dcbe-41ec-8f4b-52929ba1e348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:03.626595 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f74f6-dcbe-41ec-8f4b-52929ba1e348 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:03.627267 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7f74f6-dcbe-41ec-8f4b-52929ba1e348 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:03.629289 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7f74f6-dcbe-41ec-8f4b-52929ba1e348 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:03.761203 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1153/2291] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:33:03 2022] POST /identity/v3/projects => generated 375 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:33:03.762529 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829d90cb-09fc-48ee-891b-e3fbaa5f193e admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:03.777877 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829d90cb-09fc-48ee-891b-e3fbaa5f193e admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:03.834876 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:33:03.898763 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0401fe0-b23f-48fb-a184-df9665763f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:03.899385 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0401fe0-b23f-48fb-a184-df9665763f24 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:03.899691 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0401fe0-b23f-48fb-a184-df9665763f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:03.899978 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0401fe0-b23f-48fb-a184-df9665763f24 admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:03.900703 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0401fe0-b23f-48fb-a184-df9665763f24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:03.902737 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0401fe0-b23f-48fb-a184-df9665763f24 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:04.038535 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1139/2292] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:33:03 2022] POST /identity/v3/users => generated 288 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:33:04.041542 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f74f6-dcbe-41ec-8f4b-52929ba1e348 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:04.053488 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f74f6-dcbe-41ec-8f4b-52929ba1e348 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:04.058312 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b5ae7f-560e-4f6b-b054-b56299ece7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:04.058960 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5ae7f-560e-4f6b-b054-b56299ece7cd admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:04.059248 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5ae7f-560e-4f6b-b054-b56299ece7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:04.059551 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5ae7f-560e-4f6b-b054-b56299ece7cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:04.061001 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b5ae7f-560e-4f6b-b054-b56299ece7cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:04.071446 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b5ae7f-560e-4f6b-b054-b56299ece7cd admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:04.088117 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1154/2293] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:33:04 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:33:04.090347 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0401fe0-b23f-48fb-a184-df9665763f24 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:04.102605 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0401fe0-b23f-48fb-a184-df9665763f24 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:04.109336 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8417bd92-74e6-4474-9c3a-c1c81146ddb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:04.110511 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417bd92-74e6-4474-9c3a-c1c81146ddb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:04.110799 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417bd92-74e6-4474-9c3a-c1c81146ddb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:04.111075 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417bd92-74e6-4474-9c3a-c1c81146ddb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:04.112648 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8417bd92-74e6-4474-9c3a-c1c81146ddb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:04.115206 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8417bd92-74e6-4474-9c3a-c1c81146ddb6 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:04.134499 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1140/2294] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:33:04 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:33:04.138699 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b5ae7f-560e-4f6b-b054-b56299ece7cd admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:04.148962 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b5ae7f-560e-4f6b-b054-b56299ece7cd admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:04.153553 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5e029-17d1-4de7-8d50-16904238dbe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:04.154332 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5e029-17d1-4de7-8d50-16904238dbe2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:04.154623 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5e029-17d1-4de7-8d50-16904238dbe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:04.154907 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5e029-17d1-4de7-8d50-16904238dbe2 admin admin] PATH_INFO: `/v3/projects/ca08a54fa23349c79acb9bc02721a06d/users/5c09ca4e42f54827a45254b0dc07bc2f/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:04.182735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce5e029-17d1-4de7-8d50-16904238dbe2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca08a54fa23349c79acb9bc02721a06d, user_id=5c09ca4e42f54827a45254b0dc07bc2f, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:04.182735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce5e029-17d1-4de7-8d50-16904238dbe2 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:04.270330 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1155/2295] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:33:04 2022] PUT /identity/v3/projects/ca08a54fa23349c79acb9bc02721a06d/users/5c09ca4e42f54827a45254b0dc07bc2f/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:33:04.286444 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f535bbc-0c7b-4cb2-bb88-541e4a95953f None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:04.287697 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f535bbc-0c7b-4cb2-bb88-541e4a95953f None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:04.287996 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f535bbc-0c7b-4cb2-bb88-541e4a95953f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:04.335545 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-5f535bbc-0c7b-4cb2-bb88-541e4a95953f None None] MFA Rules not processed for user `5c09ca4e42f54827a45254b0dc07bc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:33:04.337317 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-5f535bbc-0c7b-4cb2-bb88-541e4a95953f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:04.436170 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1141/2296] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:33:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:33:04.445437 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5e029-17d1-4de7-8d50-16904238dbe2 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:04.447659 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-7ce5e029-17d1-4de7-8d50-16904238dbe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:04.532567 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5e029-17d1-4de7-8d50-16904238dbe2 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:04.540466 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b305b755-2c9b-4551-b94b-f7a4a59c6c48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:04.541126 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305b755-2c9b-4551-b94b-f7a4a59c6c48 service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:04.541454 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305b755-2c9b-4551-b94b-f7a4a59c6c48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:04.541723 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305b755-2c9b-4551-b94b-f7a4a59c6c48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:04.553643 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b305b755-2c9b-4551-b94b-f7a4a59c6c48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:04.555333 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b305b755-2c9b-4551-b94b-f7a4a59c6c48 service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:04.565764 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1156/2297] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:33:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:33:05.136907 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:33:05.182865 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:33:06.513834 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f535bbc-0c7b-4cb2-bb88-541e4a95953f None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:06.516287 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-5f535bbc-0c7b-4cb2-bb88-541e4a95953f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:06.596578 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f535bbc-0c7b-4cb2-bb88-541e4a95953f None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:06.603148 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fac7c8-2fae-4207-9f86-1c3353fe09ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:06.603931 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fac7c8-2fae-4207-9f86-1c3353fe09ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:06.604234 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fac7c8-2fae-4207-9f86-1c3353fe09ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:06.604536 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fac7c8-2fae-4207-9f86-1c3353fe09ed admin admin] PATH_INFO: `/v3/users/f2838c08a74340b98e61b6834e8753bc` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:06.605744 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fac7c8-2fae-4207-9f86-1c3353fe09ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2838c08a74340b98e61b6834e8753bc)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:06.607692 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fac7c8-2fae-4207-9f86-1c3353fe09ed admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:06.635336 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:33:06.841904 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-92fac7c8-2fae-4207-9f86-1c3353fe09ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2838c08a74340b98e61b6834e8753bc'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:06.850693 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-92fac7c8-2fae-4207-9f86-1c3353fe09ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2838c08a74340b98e61b6834e8753bc'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:06.851592 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-92fac7c8-2fae-4207-9f86-1c3353fe09ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2838c08a74340b98e61b6834e8753bc'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:06.867654 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-92fac7c8-2fae-4207-9f86-1c3353fe09ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2838c08a74340b98e61b6834e8753bc'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:06.868415 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-92fac7c8-2fae-4207-9f86-1c3353fe09ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2838c08a74340b98e61b6834e8753bc'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:06.943681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1142/2298] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:33:06 2022] DELETE /identity/v3/users/f2838c08a74340b98e61b6834e8753bc => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:33:06.948482 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f6b2bd-bb80-4523-bac9-dd6c52e7f7c2 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:06.949001 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f6b2bd-bb80-4523-bac9-dd6c52e7f7c2 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:06.949250 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f6b2bd-bb80-4523-bac9-dd6c52e7f7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:06.971170 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-38f6b2bd-bb80-4523-bac9-dd6c52e7f7c2 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:33:06.972507 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-38f6b2bd-bb80-4523-bac9-dd6c52e7f7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:07.044859 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1157/2299] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:33:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:33:07.054872 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fac7c8-2fae-4207-9f86-1c3353fe09ed admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:07.056460 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-92fac7c8-2fae-4207-9f86-1c3353fe09ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:07.117274 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fac7c8-2fae-4207-9f86-1c3353fe09ed admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:07.121009 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187045b5-671a-4c1b-a15e-94df7c7b61a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:07.121715 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187045b5-671a-4c1b-a15e-94df7c7b61a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:07.122015 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187045b5-671a-4c1b-a15e-94df7c7b61a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:07.124930 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187045b5-671a-4c1b-a15e-94df7c7b61a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:07.133827 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187045b5-671a-4c1b-a15e-94df7c7b61a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:07.135458 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187045b5-671a-4c1b-a15e-94df7c7b61a6 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:07.146032 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1143/2300] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:33:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:33:07.641582 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f6b2bd-bb80-4523-bac9-dd6c52e7f7c2 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:07.643211 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-38f6b2bd-bb80-4523-bac9-dd6c52e7f7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:07.655969 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f6b2bd-bb80-4523-bac9-dd6c52e7f7c2 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:07.660835 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15381eeb-cde7-45a6-80cc-a9ea54a14639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:07.661488 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15381eeb-cde7-45a6-80cc-a9ea54a14639 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:07.661774 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15381eeb-cde7-45a6-80cc-a9ea54a14639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:07.662056 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15381eeb-cde7-45a6-80cc-a9ea54a14639 admin admin] PATH_INFO: `/v3/projects/80965fbb5f754555b141b537a6f266d9` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:07.663382 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15381eeb-cde7-45a6-80cc-a9ea54a14639 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80965fbb5f754555b141b537a6f266d9)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:07.665361 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15381eeb-cde7-45a6-80cc-a9ea54a14639 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:07.783523 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-15381eeb-cde7-45a6-80cc-a9ea54a14639 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80965fbb5f754555b141b537a6f266d9', 'request_id': 'req-15381eeb-cde7-45a6-80cc-a9ea54a14639'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:07.789827 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-15381eeb-cde7-45a6-80cc-a9ea54a14639 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80965fbb5f754555b141b537a6f266d9', 'request_id': 'req-15381eeb-cde7-45a6-80cc-a9ea54a14639'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:07.795357 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-15381eeb-cde7-45a6-80cc-a9ea54a14639 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80965fbb5f754555b141b537a6f266d9', 'request_id': 'req-15381eeb-cde7-45a6-80cc-a9ea54a14639'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:07.838045 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-15381eeb-cde7-45a6-80cc-a9ea54a14639 admin admin] The token cache is being invalidate because project 80965fbb5f754555b141b537a6f266d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:33:07.838273 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-15381eeb-cde7-45a6-80cc-a9ea54a14639 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:07.838915 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-15381eeb-cde7-45a6-80cc-a9ea54a14639 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:07.840457 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1158/2301] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:33:07 2022] DELETE /identity/v3/projects/80965fbb5f754555b141b537a6f266d9 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:33:13.792419 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187045b5-671a-4c1b-a15e-94df7c7b61a6 service neutron] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:13.794073 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-187045b5-671a-4c1b-a15e-94df7c7b61a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:13.808765 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187045b5-671a-4c1b-a15e-94df7c7b61a6 service neutron] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:13.833519 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:33:13.867073 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751466a0-acde-4590-b9db-e143987c4c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:13.867904 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751466a0-acde-4590-b9db-e143987c4c86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:13.868549 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751466a0-acde-4590-b9db-e143987c4c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:13.869041 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751466a0-acde-4590-b9db-e143987c4c86 admin admin] PATH_INFO: `/v3/users/428e07a1a2214d129242ba76d7dc852e` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:13.882093 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751466a0-acde-4590-b9db-e143987c4c86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=428e07a1a2214d129242ba76d7dc852e)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:13.885792 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751466a0-acde-4590-b9db-e143987c4c86 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:14.105394 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-751466a0-acde-4590-b9db-e143987c4c86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '428e07a1a2214d129242ba76d7dc852e'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:14.116689 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-751466a0-acde-4590-b9db-e143987c4c86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '428e07a1a2214d129242ba76d7dc852e'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:14.117485 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-751466a0-acde-4590-b9db-e143987c4c86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '428e07a1a2214d129242ba76d7dc852e'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:14.133227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-751466a0-acde-4590-b9db-e143987c4c86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '428e07a1a2214d129242ba76d7dc852e'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:14.134068 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-751466a0-acde-4590-b9db-e143987c4c86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '428e07a1a2214d129242ba76d7dc852e'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:14.226173 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1144/2302] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:33:13 2022] DELETE /identity/v3/users/428e07a1a2214d129242ba76d7dc852e => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:33:14.230984 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4dc37-aabc-45c5-841d-1e0e4c5cfaf6 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:14.231467 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4dc37-aabc-45c5-841d-1e0e4c5cfaf6 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:14.231595 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4dc37-aabc-45c5-841d-1e0e4c5cfaf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:14.283041 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-11c4dc37-aabc-45c5-841d-1e0e4c5cfaf6 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:33:14.284446 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-11c4dc37-aabc-45c5-841d-1e0e4c5cfaf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:14.314421 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:33:14.398097 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1159/2303] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:33:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:33:14.414432 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751466a0-acde-4590-b9db-e143987c4c86 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:14.416832 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-751466a0-acde-4590-b9db-e143987c4c86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:14.507646 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751466a0-acde-4590-b9db-e143987c4c86 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:14.523287 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f645a81c-f7bb-4ac0-9103-4cb8afde0f0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:14.523287 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645a81c-f7bb-4ac0-9103-4cb8afde0f0c service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:14.523287 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645a81c-f7bb-4ac0-9103-4cb8afde0f0c service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:14.523287 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645a81c-f7bb-4ac0-9103-4cb8afde0f0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:14.541690 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f645a81c-f7bb-4ac0-9103-4cb8afde0f0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:14.543880 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f645a81c-f7bb-4ac0-9103-4cb8afde0f0c service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:14.561294 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1145/2304] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:33:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:33:15.058767 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c4dc37-aabc-45c5-841d-1e0e4c5cfaf6 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:15.065711 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-11c4dc37-aabc-45c5-841d-1e0e4c5cfaf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:15.073585 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c4dc37-aabc-45c5-841d-1e0e4c5cfaf6 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:15.092434 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8db86-b554-47a6-ae4b-990c5bd2cc13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:15.093234 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8db86-b554-47a6-ae4b-990c5bd2cc13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:15.093555 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8db86-b554-47a6-ae4b-990c5bd2cc13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:15.094942 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8db86-b554-47a6-ae4b-990c5bd2cc13 admin admin] PATH_INFO: `/v3/projects/80924db5df73440d8d549a70ded8c4b1` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:15.098193 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d8db86-b554-47a6-ae4b-990c5bd2cc13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80924db5df73440d8d549a70ded8c4b1)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:15.100158 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d8db86-b554-47a6-ae4b-990c5bd2cc13 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:15.264826 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c9d8db86-b554-47a6-ae4b-990c5bd2cc13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80924db5df73440d8d549a70ded8c4b1', 'request_id': 'req-c9d8db86-b554-47a6-ae4b-990c5bd2cc13'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:15.286358 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c9d8db86-b554-47a6-ae4b-990c5bd2cc13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80924db5df73440d8d549a70ded8c4b1', 'request_id': 'req-c9d8db86-b554-47a6-ae4b-990c5bd2cc13'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:15.292879 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c9d8db86-b554-47a6-ae4b-990c5bd2cc13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80924db5df73440d8d549a70ded8c4b1', 'request_id': 'req-c9d8db86-b554-47a6-ae4b-990c5bd2cc13'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:15.350125 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c9d8db86-b554-47a6-ae4b-990c5bd2cc13 admin admin] The token cache is being invalidate because project 80924db5df73440d8d549a70ded8c4b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:33:15.350125 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c9d8db86-b554-47a6-ae4b-990c5bd2cc13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:15.350125 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c9d8db86-b554-47a6-ae4b-990c5bd2cc13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:15.351497 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1160/2305] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:33:15 2022] DELETE /identity/v3/projects/80924db5df73440d8d549a70ded8c4b1 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:33:15.356396 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2088330-cdb8-4772-a4f7-76092f00edbc None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:15.356894 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2088330-cdb8-4772-a4f7-76092f00edbc None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:15.356986 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2088330-cdb8-4772-a4f7-76092f00edbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:15.377249 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-f2088330-cdb8-4772-a4f7-76092f00edbc None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:33:15.384476 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-f2088330-cdb8-4772-a4f7-76092f00edbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:15.444114 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:33:15.460248 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1146/2306] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:33:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:33:15.469907 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f5bc1-ab18-4a88-b316-aaa84e4ebe71 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:15.470201 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f5bc1-ab18-4a88-b316-aaa84e4ebe71 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:15.470487 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f5bc1-ab18-4a88-b316-aaa84e4ebe71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:15.490519 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-bd9f5bc1-ab18-4a88-b316-aaa84e4ebe71 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:33:15.491797 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-bd9f5bc1-ab18-4a88-b316-aaa84e4ebe71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:15.505531 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1161/2307] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:33:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:33:15.505732 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2088330-cdb8-4772-a4f7-76092f00edbc None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:15.516229 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2088330-cdb8-4772-a4f7-76092f00edbc None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:15.521135 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107f15d-4814-4e73-a587-83d88b711462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:15.521762 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107f15d-4814-4e73-a587-83d88b711462 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:15.522054 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107f15d-4814-4e73-a587-83d88b711462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:15.522337 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107f15d-4814-4e73-a587-83d88b711462 admin admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:15.523043 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5107f15d-4814-4e73-a587-83d88b711462 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:33:15.523362 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5107f15d-4814-4e73-a587-83d88b711462 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:15.525148 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5107f15d-4814-4e73-a587-83d88b711462 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:15.541548 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1147/2308] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:33:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:33:15.545822 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f5bc1-ab18-4a88-b316-aaa84e4ebe71 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:15.557316 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f5bc1-ab18-4a88-b316-aaa84e4ebe71 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:15.562773 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2864b54-5f06-4362-bd21-8cb032710ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:15.564122 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2864b54-5f06-4362-bd21-8cb032710ede admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:15.564413 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2864b54-5f06-4362-bd21-8cb032710ede admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:15.564694 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2864b54-5f06-4362-bd21-8cb032710ede admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:15.565372 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2864b54-5f06-4362-bd21-8cb032710ede admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:15.567298 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2864b54-5f06-4362-bd21-8cb032710ede admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:15.646062 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1162/2309] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:33:15 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:33:15.647426 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107f15d-4814-4e73-a587-83d88b711462 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:15.658064 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107f15d-4814-4e73-a587-83d88b711462 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:15.745453 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a295fa9c-884d-476a-a791-0d75e9a5f2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:15.746475 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a295fa9c-884d-476a-a791-0d75e9a5f2f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:15.746937 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a295fa9c-884d-476a-a791-0d75e9a5f2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:15.747467 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a295fa9c-884d-476a-a791-0d75e9a5f2f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:15.748496 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a295fa9c-884d-476a-a791-0d75e9a5f2f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:15.750729 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a295fa9c-884d-476a-a791-0d75e9a5f2f3 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:15.819581 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1148/2310] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:33:15 2022] POST /identity/v3/users => generated 288 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:33:15.824118 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2864b54-5f06-4362-bd21-8cb032710ede admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:15.845775 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2864b54-5f06-4362-bd21-8cb032710ede admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:15.851390 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2535b2b-dce9-4ae7-8db0-08bd5974b221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:15.852027 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2535b2b-dce9-4ae7-8db0-08bd5974b221 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:15.852513 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2535b2b-dce9-4ae7-8db0-08bd5974b221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:15.853022 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2535b2b-dce9-4ae7-8db0-08bd5974b221 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:15.853842 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2535b2b-dce9-4ae7-8db0-08bd5974b221 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:15.855661 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2535b2b-dce9-4ae7-8db0-08bd5974b221 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:15.884102 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1163/2311] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:33:15 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:33:15.886300 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a295fa9c-884d-476a-a791-0d75e9a5f2f3 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:15.898249 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a295fa9c-884d-476a-a791-0d75e9a5f2f3 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:15.906639 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7341d765-a47c-4e72-b261-125709b13a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:15.907348 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341d765-a47c-4e72-b261-125709b13a56 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:15.907632 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341d765-a47c-4e72-b261-125709b13a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:15.907907 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341d765-a47c-4e72-b261-125709b13a56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:15.908647 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7341d765-a47c-4e72-b261-125709b13a56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:15.910283 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7341d765-a47c-4e72-b261-125709b13a56 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:15.922204 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1149/2312] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:33:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 15:33:15.926443 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2535b2b-dce9-4ae7-8db0-08bd5974b221 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:15.939535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2535b2b-dce9-4ae7-8db0-08bd5974b221 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:15.945036 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42ef945-3d2f-46ef-b009-49e3b2326caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:15.945841 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ef945-3d2f-46ef-b009-49e3b2326caa admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:15.946235 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ef945-3d2f-46ef-b009-49e3b2326caa admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:15.946587 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ef945-3d2f-46ef-b009-49e3b2326caa admin admin] PATH_INFO: `/v3/projects/1949265777ca48ce96416be1a8ca67ec/users/b1ad72480c184986be0260b8c4323d91/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:15.965676 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42ef945-3d2f-46ef-b009-49e3b2326caa admin admin] RBAC: Authorizing `identity:create_grant(project_id=1949265777ca48ce96416be1a8ca67ec, user_id=b1ad72480c184986be0260b8c4323d91, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:15.969346 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42ef945-3d2f-46ef-b009-49e3b2326caa admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:16.060831 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1164/2313] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:33:15 2022] PUT /identity/v3/projects/1949265777ca48ce96416be1a8ca67ec/users/b1ad72480c184986be0260b8c4323d91/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:33:16.071829 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e71336-3efb-49cc-9408-134e24b87327 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:16.072176 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e71336-3efb-49cc-9408-134e24b87327 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:16.073081 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e71336-3efb-49cc-9408-134e24b87327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:16.106443 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-83e71336-3efb-49cc-9408-134e24b87327 None None] MFA Rules not processed for user `b1ad72480c184986be0260b8c4323d91`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:33:16.108029 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-83e71336-3efb-49cc-9408-134e24b87327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:16.235876 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1150/2314] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:33:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:33:16.249200 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42ef945-3d2f-46ef-b009-49e3b2326caa admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:16.251798 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-d42ef945-3d2f-46ef-b009-49e3b2326caa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:16.317891 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:33:16.339672 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42ef945-3d2f-46ef-b009-49e3b2326caa admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:16.344815 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f58230-bfd7-4a2c-9bec-449c33dd692a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:16.345462 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f58230-bfd7-4a2c-9bec-449c33dd692a service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:16.345752 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f58230-bfd7-4a2c-9bec-449c33dd692a service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:16.346034 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f58230-bfd7-4a2c-9bec-449c33dd692a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:16.361483 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f58230-bfd7-4a2c-9bec-449c33dd692a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:16.364193 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f58230-bfd7-4a2c-9bec-449c33dd692a service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:16.392263 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1165/2315] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:33:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:33:18.688556 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e71336-3efb-49cc-9408-134e24b87327 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:18.690071 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-83e71336-3efb-49cc-9408-134e24b87327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:18.740257 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:33:18.759897 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e71336-3efb-49cc-9408-134e24b87327 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:18.764698 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d505fe-9cd0-4981-aa2d-f8af0c96e2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:18.765354 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d505fe-9cd0-4981-aa2d-f8af0c96e2c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:18.765658 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d505fe-9cd0-4981-aa2d-f8af0c96e2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:18.765939 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d505fe-9cd0-4981-aa2d-f8af0c96e2c5 admin admin] PATH_INFO: `/v3/users/5c09ca4e42f54827a45254b0dc07bc2f` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:18.767218 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d505fe-9cd0-4981-aa2d-f8af0c96e2c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c09ca4e42f54827a45254b0dc07bc2f)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:18.769258 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d505fe-9cd0-4981-aa2d-f8af0c96e2c5 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:18.924166 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-b4d505fe-9cd0-4981-aa2d-f8af0c96e2c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c09ca4e42f54827a45254b0dc07bc2f'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:18.931795 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-b4d505fe-9cd0-4981-aa2d-f8af0c96e2c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c09ca4e42f54827a45254b0dc07bc2f'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:18.932509 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-b4d505fe-9cd0-4981-aa2d-f8af0c96e2c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c09ca4e42f54827a45254b0dc07bc2f'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:18.944859 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-b4d505fe-9cd0-4981-aa2d-f8af0c96e2c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c09ca4e42f54827a45254b0dc07bc2f'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:18.945535 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-b4d505fe-9cd0-4981-aa2d-f8af0c96e2c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c09ca4e42f54827a45254b0dc07bc2f'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:19.028693 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1151/2316] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:33:18 2022] DELETE /identity/v3/users/5c09ca4e42f54827a45254b0dc07bc2f => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:33:19.033821 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8fad28-4ca8-4d6a-88f0-882c994b2b12 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:19.034281 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8fad28-4ca8-4d6a-88f0-882c994b2b12 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:19.034397 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8fad28-4ca8-4d6a-88f0-882c994b2b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:19.053232 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-ff8fad28-4ca8-4d6a-88f0-882c994b2b12 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:33:19.054419 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-ff8fad28-4ca8-4d6a-88f0-882c994b2b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:19.112015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1166/2317] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:33:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:33:19.121856 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d505fe-9cd0-4981-aa2d-f8af0c96e2c5 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:19.123338 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-b4d505fe-9cd0-4981-aa2d-f8af0c96e2c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:19.181201 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d505fe-9cd0-4981-aa2d-f8af0c96e2c5 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:19.184908 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524396c-2475-43f7-abd2-80fded3523d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:19.185528 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524396c-2475-43f7-abd2-80fded3523d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:19.185812 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524396c-2475-43f7-abd2-80fded3523d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:19.186090 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524396c-2475-43f7-abd2-80fded3523d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:19.195296 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a524396c-2475-43f7-abd2-80fded3523d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:19.196780 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a524396c-2475-43f7-abd2-80fded3523d5 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:19.208116 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1152/2318] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:33:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:33:19.648219 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8fad28-4ca8-4d6a-88f0-882c994b2b12 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:19.649676 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-ff8fad28-4ca8-4d6a-88f0-882c994b2b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:19.663063 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8fad28-4ca8-4d6a-88f0-882c994b2b12 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:19.668623 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa08586c-b41e-4768-9d46-598c2a9b22b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:19.669272 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08586c-b41e-4768-9d46-598c2a9b22b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:19.669560 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08586c-b41e-4768-9d46-598c2a9b22b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:19.669837 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08586c-b41e-4768-9d46-598c2a9b22b8 admin admin] PATH_INFO: `/v3/projects/ca08a54fa23349c79acb9bc02721a06d` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:19.671072 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa08586c-b41e-4768-9d46-598c2a9b22b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca08a54fa23349c79acb9bc02721a06d)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:19.673661 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa08586c-b41e-4768-9d46-598c2a9b22b8 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:19.783203 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-fa08586c-b41e-4768-9d46-598c2a9b22b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca08a54fa23349c79acb9bc02721a06d', 'request_id': 'req-fa08586c-b41e-4768-9d46-598c2a9b22b8'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:19.789429 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-fa08586c-b41e-4768-9d46-598c2a9b22b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca08a54fa23349c79acb9bc02721a06d', 'request_id': 'req-fa08586c-b41e-4768-9d46-598c2a9b22b8'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:19.795284 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-fa08586c-b41e-4768-9d46-598c2a9b22b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca08a54fa23349c79acb9bc02721a06d', 'request_id': 'req-fa08586c-b41e-4768-9d46-598c2a9b22b8'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:19.891057 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-fa08586c-b41e-4768-9d46-598c2a9b22b8 admin admin] The token cache is being invalidate because project ca08a54fa23349c79acb9bc02721a06d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:33:19.891350 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-fa08586c-b41e-4768-9d46-598c2a9b22b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:19.891971 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-fa08586c-b41e-4768-9d46-598c2a9b22b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:19.893505 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1167/2319] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:33:19 2022] DELETE /identity/v3/projects/ca08a54fa23349c79acb9bc02721a06d => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:33:19.898267 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c737f9e0-6ddf-469a-b15d-83f0eb1c9ce5 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:19.898742 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c737f9e0-6ddf-469a-b15d-83f0eb1c9ce5 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:19.898839 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c737f9e0-6ddf-469a-b15d-83f0eb1c9ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:19.918372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-c737f9e0-6ddf-469a-b15d-83f0eb1c9ce5 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:33:19.919642 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-c737f9e0-6ddf-469a-b15d-83f0eb1c9ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:19.982676 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1153/2320] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:33:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:33:19.992035 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642addcc-0bce-44ae-be27-964612e86516 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:19.995505 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642addcc-0bce-44ae-be27-964612e86516 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:19.995505 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642addcc-0bce-44ae-be27-964612e86516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:20.013653 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-642addcc-0bce-44ae-be27-964612e86516 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:33:20.014857 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-642addcc-0bce-44ae-be27-964612e86516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:20.023243 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1168/2321] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:33:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:33:20.027497 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c737f9e0-6ddf-469a-b15d-83f0eb1c9ce5 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:20.038126 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c737f9e0-6ddf-469a-b15d-83f0eb1c9ce5 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:20.042754 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e51dd7-e654-4647-b78c-d91d38c95a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:20.043523 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e51dd7-e654-4647-b78c-d91d38c95a01 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:20.043642 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e51dd7-e654-4647-b78c-d91d38c95a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:20.043896 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e51dd7-e654-4647-b78c-d91d38c95a01 admin admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:20.044618 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e51dd7-e654-4647-b78c-d91d38c95a01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:33:20.044921 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e51dd7-e654-4647-b78c-d91d38c95a01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:20.046546 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e51dd7-e654-4647-b78c-d91d38c95a01 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:20.048687 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:33:20.059494 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1154/2322] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:33:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:33:20.064007 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642addcc-0bce-44ae-be27-964612e86516 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:20.073731 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642addcc-0bce-44ae-be27-964612e86516 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:20.078535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04533d92-313d-46b8-b7f0-7e02a343ef60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:20.079175 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04533d92-313d-46b8-b7f0-7e02a343ef60 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:20.079458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04533d92-313d-46b8-b7f0-7e02a343ef60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:20.079739 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04533d92-313d-46b8-b7f0-7e02a343ef60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:20.080435 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04533d92-313d-46b8-b7f0-7e02a343ef60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:20.082327 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04533d92-313d-46b8-b7f0-7e02a343ef60 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:20.190284 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1169/2323] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:33:20 2022] POST /identity/v3/projects => generated 391 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:33:20.194622 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e51dd7-e654-4647-b78c-d91d38c95a01 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:20.204768 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e51dd7-e654-4647-b78c-d91d38c95a01 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:20.261298 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067a04c-7421-45d1-ab92-25eb238b7956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:20.261915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067a04c-7421-45d1-ab92-25eb238b7956 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:20.262203 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067a04c-7421-45d1-ab92-25eb238b7956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:20.262482 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067a04c-7421-45d1-ab92-25eb238b7956 admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:20.263154 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b067a04c-7421-45d1-ab92-25eb238b7956 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:20.265116 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b067a04c-7421-45d1-ab92-25eb238b7956 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:20.341759 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1155/2324] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:33:20 2022] POST /identity/v3/users => generated 296 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:33:20.346217 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04533d92-313d-46b8-b7f0-7e02a343ef60 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:20.356771 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04533d92-313d-46b8-b7f0-7e02a343ef60 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:20.362058 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2941c125-b7b0-4471-88d4-a5d3667b673d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:20.362663 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941c125-b7b0-4471-88d4-a5d3667b673d admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:20.362947 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941c125-b7b0-4471-88d4-a5d3667b673d admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:20.363228 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941c125-b7b0-4471-88d4-a5d3667b673d admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:20.363921 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2941c125-b7b0-4471-88d4-a5d3667b673d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:20.365809 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2941c125-b7b0-4471-88d4-a5d3667b673d admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:20.375683 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1170/2325] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:33:20.379495 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067a04c-7421-45d1-ab92-25eb238b7956 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:20.389996 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067a04c-7421-45d1-ab92-25eb238b7956 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:20.395234 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4570d3-0571-4bd8-aef6-40aa25cd5869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:20.395830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4570d3-0571-4bd8-aef6-40aa25cd5869 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:20.396145 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4570d3-0571-4bd8-aef6-40aa25cd5869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:20.396424 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4570d3-0571-4bd8-aef6-40aa25cd5869 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:20.397121 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4570d3-0571-4bd8-aef6-40aa25cd5869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:20.398818 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4570d3-0571-4bd8-aef6-40aa25cd5869 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:20.409906 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1156/2326] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:33:20.413838 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2941c125-b7b0-4471-88d4-a5d3667b673d admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:20.423766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2941c125-b7b0-4471-88d4-a5d3667b673d admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:20.428373 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca93ee-4405-45ea-8db5-0e5f46ea96e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:20.429025 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca93ee-4405-45ea-8db5-0e5f46ea96e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:20.429309 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca93ee-4405-45ea-8db5-0e5f46ea96e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:20.429591 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca93ee-4405-45ea-8db5-0e5f46ea96e5 admin admin] PATH_INFO: `/v3/projects/a0799a1305fe48c19195af4a33844448/users/286696e0336847cba2a7858a47d167cc/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:20.443928 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca93ee-4405-45ea-8db5-0e5f46ea96e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0799a1305fe48c19195af4a33844448, user_id=286696e0336847cba2a7858a47d167cc, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:20.446592 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca93ee-4405-45ea-8db5-0e5f46ea96e5 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:20.503932 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1171/2327] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:33:20 2022] PUT /identity/v3/projects/a0799a1305fe48c19195af4a33844448/users/286696e0336847cba2a7858a47d167cc/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:33:20.514307 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee79d11-af9b-4d60-8618-5ed094b3b257 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:20.514528 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee79d11-af9b-4d60-8618-5ed094b3b257 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:20.514806 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee79d11-af9b-4d60-8618-5ed094b3b257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:20.545808 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-3ee79d11-af9b-4d60-8618-5ed094b3b257 None None] MFA Rules not processed for user `286696e0336847cba2a7858a47d167cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:33:20.547023 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-3ee79d11-af9b-4d60-8618-5ed094b3b257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:20.614299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1157/2328] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:33:20 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:33:20.630116 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca93ee-4405-45ea-8db5-0e5f46ea96e5 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:20.631894 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-b0ca93ee-4405-45ea-8db5-0e5f46ea96e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:20.643701 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:33:20.705332 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca93ee-4405-45ea-8db5-0e5f46ea96e5 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:20.709584 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c91873-618d-418a-a834-961bb81f6988 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:20.710583 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c91873-618d-418a-a834-961bb81f6988 service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:20.710997 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c91873-618d-418a-a834-961bb81f6988 service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:20.711498 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c91873-618d-418a-a834-961bb81f6988 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:20.723646 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c91873-618d-418a-a834-961bb81f6988 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:20.725157 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c91873-618d-418a-a834-961bb81f6988 service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:20.734735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1172/2329] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:33:20 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:33:30.611208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=69,keystone:INSERT=3 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:33:30.840543 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=22 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:33:31.354737 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9f21c-0d6f-41f6-86ab-cc0e8c57ab62 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:31.355087 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9f21c-0d6f-41f6-86ab-cc0e8c57ab62 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:31.355370 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9f21c-0d6f-41f6-86ab-cc0e8c57ab62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:31.377180 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-69a9f21c-0d6f-41f6-86ab-cc0e8c57ab62 None None] MFA Rules not processed for user `1782b4e84c9f484297e17c248738105c`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:33:31.378547 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-69a9f21c-0d6f-41f6-86ab-cc0e8c57ab62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:31.438456 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1158/2330] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:33:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:33:31.487662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f54af-fb6e-4e08-9738-484204cb2ef7 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:31.488013 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f54af-fb6e-4e08-9738-484204cb2ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:31.488250 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f54af-fb6e-4e08-9738-484204cb2ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:31.510235 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-328f54af-fb6e-4e08-9738-484204cb2ef7 None None] MFA Rules not processed for user `1654fb6a81e14d6b99ab495bb6d029d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:33:31.511466 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-328f54af-fb6e-4e08-9738-484204cb2ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:31.572668 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1173/2331] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 5 15:33:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:33:31.616547 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9f21c-0d6f-41f6-86ab-cc0e8c57ab62 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:31.618349 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-69a9f21c-0d6f-41f6-86ab-cc0e8c57ab62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:31.701925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9f21c-0d6f-41f6-86ab-cc0e8c57ab62 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:31.708337 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee025569-e49a-4d7c-987d-33fa1905743a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85154ce7446245738edcc8702dbf6861', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:31.708971 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee025569-e49a-4d7c-987d-33fa1905743a service swift] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:31.709248 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee025569-e49a-4d7c-987d-33fa1905743a service swift] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:31.709530 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee025569-e49a-4d7c-987d-33fa1905743a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:31.718676 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee025569-e49a-4d7c-987d-33fa1905743a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:31.720978 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee025569-e49a-4d7c-987d-33fa1905743a service swift] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:31.730104 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1159/2332] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 5 15:33:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 15:33:41.565529 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=25 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:33:41.727015 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=57 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:33:47.060754 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328f54af-fb6e-4e08-9738-484204cb2ef7 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:47.126570 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328f54af-fb6e-4e08-9738-484204cb2ef7 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:47.131752 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4965d61c-68c1-437e-82ed-c021007719e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:47.132433 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4965d61c-68c1-437e-82ed-c021007719e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:47.132714 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4965d61c-68c1-437e-82ed-c021007719e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:47.132993 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4965d61c-68c1-437e-82ed-c021007719e1 admin admin] PATH_INFO: `/v3/users/286696e0336847cba2a7858a47d167cc` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:47.134271 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4965d61c-68c1-437e-82ed-c021007719e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=286696e0336847cba2a7858a47d167cc)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:47.136417 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4965d61c-68c1-437e-82ed-c021007719e1 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:47.302540 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-4965d61c-68c1-437e-82ed-c021007719e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '286696e0336847cba2a7858a47d167cc'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:47.303727 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-4965d61c-68c1-437e-82ed-c021007719e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '286696e0336847cba2a7858a47d167cc'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:47.323045 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-4965d61c-68c1-437e-82ed-c021007719e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '286696e0336847cba2a7858a47d167cc'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:47.409951 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-4965d61c-68c1-437e-82ed-c021007719e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '286696e0336847cba2a7858a47d167cc'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:47.418347 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-4965d61c-68c1-437e-82ed-c021007719e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '286696e0336847cba2a7858a47d167cc'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:47.424973 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1174/2333] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:33:47 2022] DELETE /identity/v3/users/286696e0336847cba2a7858a47d167cc => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:33:47.429480 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e876026-ebd9-41e1-ad48-656a15ee84f1 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:47.429951 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e876026-ebd9-41e1-ad48-656a15ee84f1 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:47.430072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e876026-ebd9-41e1-ad48-656a15ee84f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:47.449871 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-3e876026-ebd9-41e1-ad48-656a15ee84f1 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:33:47.453269 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-3e876026-ebd9-41e1-ad48-656a15ee84f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:47.518238 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1160/2334] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:33:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:33:47.529012 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4965d61c-68c1-437e-82ed-c021007719e1 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:47.530533 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-4965d61c-68c1-437e-82ed-c021007719e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:47.603313 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4965d61c-68c1-437e-82ed-c021007719e1 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:47.607379 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28457d7e-343a-4558-889f-727d489f368b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:47.608008 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28457d7e-343a-4558-889f-727d489f368b service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:47.608316 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28457d7e-343a-4558-889f-727d489f368b service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:47.608593 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28457d7e-343a-4558-889f-727d489f368b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:47.617880 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28457d7e-343a-4558-889f-727d489f368b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:47.619310 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28457d7e-343a-4558-889f-727d489f368b service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:47.629174 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1175/2335] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:33:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:33:48.121712 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e876026-ebd9-41e1-ad48-656a15ee84f1 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:48.123188 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-3e876026-ebd9-41e1-ad48-656a15ee84f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:48.137989 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e876026-ebd9-41e1-ad48-656a15ee84f1 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:48.142912 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57544d41-75da-4737-a3cd-09a7ece2292a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:48.143567 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57544d41-75da-4737-a3cd-09a7ece2292a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:48.143848 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57544d41-75da-4737-a3cd-09a7ece2292a admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:48.144148 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57544d41-75da-4737-a3cd-09a7ece2292a admin admin] PATH_INFO: `/v3/projects/a0799a1305fe48c19195af4a33844448` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:48.145321 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57544d41-75da-4737-a3cd-09a7ece2292a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0799a1305fe48c19195af4a33844448)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:48.148497 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57544d41-75da-4737-a3cd-09a7ece2292a admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:48.251542 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-57544d41-75da-4737-a3cd-09a7ece2292a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0799a1305fe48c19195af4a33844448', 'request_id': 'req-57544d41-75da-4737-a3cd-09a7ece2292a'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:48.258344 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-57544d41-75da-4737-a3cd-09a7ece2292a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0799a1305fe48c19195af4a33844448', 'request_id': 'req-57544d41-75da-4737-a3cd-09a7ece2292a'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:48.312975 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-57544d41-75da-4737-a3cd-09a7ece2292a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0799a1305fe48c19195af4a33844448', 'request_id': 'req-57544d41-75da-4737-a3cd-09a7ece2292a'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:48.325721 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-57544d41-75da-4737-a3cd-09a7ece2292a admin admin] The token cache is being invalidate because project a0799a1305fe48c19195af4a33844448 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:33:48.326011 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-57544d41-75da-4737-a3cd-09a7ece2292a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:48.326643 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-57544d41-75da-4737-a3cd-09a7ece2292a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:48.328180 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1161/2336] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:33:48 2022] DELETE /identity/v3/projects/a0799a1305fe48c19195af4a33844448 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:33:48.696090 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28457d7e-343a-4558-889f-727d489f368b service neutron] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:48.697598 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-28457d7e-343a-4558-889f-727d489f368b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:48.710657 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28457d7e-343a-4558-889f-727d489f368b service neutron] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:48.730368 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:33:48.768377 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23eec0a-f2db-40fa-bad7-1367f9a44899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:48.769028 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23eec0a-f2db-40fa-bad7-1367f9a44899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:48.769320 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23eec0a-f2db-40fa-bad7-1367f9a44899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:48.769606 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23eec0a-f2db-40fa-bad7-1367f9a44899 admin admin] PATH_INFO: `/v3/users/b1ad72480c184986be0260b8c4323d91` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:48.770788 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23eec0a-f2db-40fa-bad7-1367f9a44899 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1ad72480c184986be0260b8c4323d91)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:48.772811 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23eec0a-f2db-40fa-bad7-1367f9a44899 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:48.956732 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-a23eec0a-f2db-40fa-bad7-1367f9a44899 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1ad72480c184986be0260b8c4323d91'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:48.958023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-a23eec0a-f2db-40fa-bad7-1367f9a44899 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1ad72480c184986be0260b8c4323d91'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:48.970520 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-a23eec0a-f2db-40fa-bad7-1367f9a44899 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1ad72480c184986be0260b8c4323d91'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:49.044850 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-a23eec0a-f2db-40fa-bad7-1367f9a44899 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1ad72480c184986be0260b8c4323d91'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:49.052161 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-a23eec0a-f2db-40fa-bad7-1367f9a44899 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1ad72480c184986be0260b8c4323d91'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:49.058408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1176/2337] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:33:48 2022] DELETE /identity/v3/users/b1ad72480c184986be0260b8c4323d91 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:33:49.063224 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba071fb-7b19-411a-a5aa-4241aac9b31c None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:49.063720 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba071fb-7b19-411a-a5aa-4241aac9b31c None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:49.063943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba071fb-7b19-411a-a5aa-4241aac9b31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:49.082825 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-eba071fb-7b19-411a-a5aa-4241aac9b31c None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:33:49.084681 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-eba071fb-7b19-411a-a5aa-4241aac9b31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:49.143039 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1162/2338] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:33:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:33:49.153213 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23eec0a-f2db-40fa-bad7-1367f9a44899 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:49.154847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-a23eec0a-f2db-40fa-bad7-1367f9a44899 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:49.216138 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23eec0a-f2db-40fa-bad7-1367f9a44899 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:49.219871 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbe13ba-c1c1-49f1-93d6-dd47375b4a21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:49.220547 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe13ba-c1c1-49f1-93d6-dd47375b4a21 service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:49.220821 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe13ba-c1c1-49f1-93d6-dd47375b4a21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:49.221096 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe13ba-c1c1-49f1-93d6-dd47375b4a21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:49.231779 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbe13ba-c1c1-49f1-93d6-dd47375b4a21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:49.233289 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbe13ba-c1c1-49f1-93d6-dd47375b4a21 service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:49.243373 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1177/2339] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:33:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 15:33:49.281839 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593eb1d-c9ec-4aa6-ad09-faa9d4710f20 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:49.282132 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593eb1d-c9ec-4aa6-ad09-faa9d4710f20 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:49.282417 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593eb1d-c9ec-4aa6-ad09-faa9d4710f20 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:49.283865 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1163/2340] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 5 15:33:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:33:49.289813 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca66d43-23d9-48b4-a231-ea5809a41438 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:49.290109 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca66d43-23d9-48b4-a231-ea5809a41438 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:49.290486 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca66d43-23d9-48b4-a231-ea5809a41438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:49.310596 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-3ca66d43-23d9-48b4-a231-ea5809a41438 None None] MFA Rules not processed for user `bc0497dab1f44bb7b8196d39bf01d770`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:33:49.311842 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-3ca66d43-23d9-48b4-a231-ea5809a41438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:49.356823 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:33:49.384358 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1178/2341] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 5 15:33:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:33:49.395927 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593eb1d-c9ec-4aa6-ad09-faa9d4710f20 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:49.398313 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-0593eb1d-c9ec-4aa6-ad09-faa9d4710f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:49.426489 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:33:49.479548 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593eb1d-c9ec-4aa6-ad09-faa9d4710f20 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:49.483452 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66d5fdd-4a51-46c0-b36f-e649cca1e4d2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c386fa0483cf42efaddbc2395f6d1fce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:49.488527 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d5fdd-4a51-46c0-b36f-e649cca1e4d2 service glance] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:49.488527 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d5fdd-4a51-46c0-b36f-e649cca1e4d2 service glance] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:49.488527 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d5fdd-4a51-46c0-b36f-e649cca1e4d2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:49.496953 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66d5fdd-4a51-46c0-b36f-e649cca1e4d2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:49.498401 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66d5fdd-4a51-46c0-b36f-e649cca1e4d2 service glance] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:49.510228 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1164/2342] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:33:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:33:49.698436 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab64b4d-3d67-407f-aed6-db8f6ddd84c5 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:49.698973 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab64b4d-3d67-407f-aed6-db8f6ddd84c5 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:49.699068 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab64b4d-3d67-407f-aed6-db8f6ddd84c5 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:49.700436 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1179/2343] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 5 15:33:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:33:49.705176 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c769d-eca3-4b0a-b7f6-e472bb5a797d None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:49.705467 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c769d-eca3-4b0a-b7f6-e472bb5a797d None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:49.705746 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c769d-eca3-4b0a-b7f6-e472bb5a797d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:49.725318 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-5a1c769d-eca3-4b0a-b7f6-e472bb5a797d None None] MFA Rules not processed for user `d13a1638b2ca497fb02c71615559d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:33:49.726522 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-5a1c769d-eca3-4b0a-b7f6-e472bb5a797d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:49.733682 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1165/2344] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 5 15:33:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:33:49.746229 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab64b4d-3d67-407f-aed6-db8f6ddd84c5 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:49.748041 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-4ab64b4d-3d67-407f-aed6-db8f6ddd84c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:49.754486 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c769d-eca3-4b0a-b7f6-e472bb5a797d None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:49.756924 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-5a1c769d-eca3-4b0a-b7f6-e472bb5a797d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:49.763473 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab64b4d-3d67-407f-aed6-db8f6ddd84c5 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:49.768434 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31c39ce-403e-44c7-a78c-9ca42ea3a36f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:49.769766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c39ce-403e-44c7-a78c-9ca42ea3a36f service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:49.770351 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c39ce-403e-44c7-a78c-9ca42ea3a36f service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:49.770525 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c39ce-403e-44c7-a78c-9ca42ea3a36f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:49.771770 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c769d-eca3-4b0a-b7f6-e472bb5a797d None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:49.777098 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade9a42-50cc-448d-a61a-ea2ebd8322e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:49.777884 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade9a42-50cc-448d-a61a-ea2ebd8322e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:49.778200 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade9a42-50cc-448d-a61a-ea2ebd8322e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:49.778599 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade9a42-50cc-448d-a61a-ea2ebd8322e9 admin admin] PATH_INFO: `/v3/projects/1949265777ca48ce96416be1a8ca67ec` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:49.779441 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31c39ce-403e-44c7-a78c-9ca42ea3a36f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:49.779902 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade9a42-50cc-448d-a61a-ea2ebd8322e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1949265777ca48ce96416be1a8ca67ec)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:49.781111 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31c39ce-403e-44c7-a78c-9ca42ea3a36f service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:49.782158 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade9a42-50cc-448d-a61a-ea2ebd8322e9 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:49.792011 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1180/2345] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:33:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:33:49.875829 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3934549-5a94-48dd-a3cd-e36837e63c38 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:49.876140 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3934549-5a94-48dd-a3cd-e36837e63c38 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:49.876423 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3934549-5a94-48dd-a3cd-e36837e63c38 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:49.877787 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1181/2346] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 5 15:33:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:33:49.883673 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5996a3-b866-46a3-b314-1d466acd7083 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:49.883956 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5996a3-b866-46a3-b314-1d466acd7083 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:49.884268 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5996a3-b866-46a3-b314-1d466acd7083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:49.893932 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-bade9a42-50cc-448d-a61a-ea2ebd8322e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1949265777ca48ce96416be1a8ca67ec', 'request_id': 'req-bade9a42-50cc-448d-a61a-ea2ebd8322e9'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:49.900265 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-bade9a42-50cc-448d-a61a-ea2ebd8322e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1949265777ca48ce96416be1a8ca67ec', 'request_id': 'req-bade9a42-50cc-448d-a61a-ea2ebd8322e9'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:49.903581 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-6a5996a3-b866-46a3-b314-1d466acd7083 None None] MFA Rules not processed for user `bc0497dab1f44bb7b8196d39bf01d770`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:33:49.904827 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-6a5996a3-b866-46a3-b314-1d466acd7083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:49.971770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1182/2347] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 5 15:33:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:33:49.976887 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-bade9a42-50cc-448d-a61a-ea2ebd8322e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1949265777ca48ce96416be1a8ca67ec', 'request_id': 'req-bade9a42-50cc-448d-a61a-ea2ebd8322e9'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:49.983144 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5996a3-b866-46a3-b314-1d466acd7083 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:49.988324 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-bade9a42-50cc-448d-a61a-ea2ebd8322e9 admin admin] The token cache is being invalidate because project 1949265777ca48ce96416be1a8ca67ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:33:49.988678 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-bade9a42-50cc-448d-a61a-ea2ebd8322e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:49.991633 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-bade9a42-50cc-448d-a61a-ea2ebd8322e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:49.991633 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1166/2348] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:33:49 2022] DELETE /identity/v3/projects/1949265777ca48ce96416be1a8ca67ec => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:33:49.997114 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9db3bd-ae8d-4dcd-8c5f-08fa27b260f6 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:49.997381 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9db3bd-ae8d-4dcd-8c5f-08fa27b260f6 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:49.997665 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9db3bd-ae8d-4dcd-8c5f-08fa27b260f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:50.017187 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-4f9db3bd-ae8d-4dcd-8c5f-08fa27b260f6 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:33:50.018626 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-4f9db3bd-ae8d-4dcd-8c5f-08fa27b260f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:50.039586 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5996a3-b866-46a3-b314-1d466acd7083 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:50.045492 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbefbe7-55b8-4ef4-b60d-4fb01d1e736c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:50.046133 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbefbe7-55b8-4ef4-b60d-4fb01d1e736c service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:50.046414 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbefbe7-55b8-4ef4-b60d-4fb01d1e736c service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:50.046693 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbefbe7-55b8-4ef4-b60d-4fb01d1e736c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:50.048689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-fcbefbe7-55b8-4ef4-b60d-4fb01d1e736c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:50.068683 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbefbe7-55b8-4ef4-b60d-4fb01d1e736c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:50.070389 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbefbe7-55b8-4ef4-b60d-4fb01d1e736c service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:50.081184 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1183/2349] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:33:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 15:33:50.087177 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1167/2350] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:33:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:33:50.096521 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363ad64-c7b1-4ac4-9417-01fca5709490 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:50.096813 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363ad64-c7b1-4ac4-9417-01fca5709490 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:50.097095 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363ad64-c7b1-4ac4-9417-01fca5709490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:50.117425 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-4363ad64-c7b1-4ac4-9417-01fca5709490 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:33:50.118621 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-4363ad64-c7b1-4ac4-9417-01fca5709490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:50.126960 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1184/2351] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:33:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:33:50.131468 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9db3bd-ae8d-4dcd-8c5f-08fa27b260f6 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:50.141514 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9db3bd-ae8d-4dcd-8c5f-08fa27b260f6 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:50.146418 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd12f7-9f03-4639-85ff-79f72175c6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:50.146972 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd12f7-9f03-4639-85ff-79f72175c6d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:50.147259 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd12f7-9f03-4639-85ff-79f72175c6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:50.147540 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd12f7-9f03-4639-85ff-79f72175c6d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:50.148273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dd12f7-9f03-4639-85ff-79f72175c6d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:33:50.148589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dd12f7-9f03-4639-85ff-79f72175c6d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:50.150177 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dd12f7-9f03-4639-85ff-79f72175c6d0 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:50.162067 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1168/2352] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:33:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:33:50.168388 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4363ad64-c7b1-4ac4-9417-01fca5709490 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:50.176864 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4363ad64-c7b1-4ac4-9417-01fca5709490 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:50.182210 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6263c1e1-19b7-4829-b839-37980f534c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:50.182823 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263c1e1-19b7-4829-b839-37980f534c08 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:50.183167 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263c1e1-19b7-4829-b839-37980f534c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:50.183472 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263c1e1-19b7-4829-b839-37980f534c08 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:50.184176 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6263c1e1-19b7-4829-b839-37980f534c08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:50.186087 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6263c1e1-19b7-4829-b839-37980f534c08 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:50.264108 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1185/2353] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:33:50 2022] POST /identity/v3/projects => generated 371 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:33:50.276106 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd12f7-9f03-4639-85ff-79f72175c6d0 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:50.278069 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd12f7-9f03-4639-85ff-79f72175c6d0 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:50.286100 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:33:50.336638 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65e63c-9c94-451b-a985-20f7090f5c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:50.337258 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65e63c-9c94-451b-a985-20f7090f5c39 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:50.337544 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65e63c-9c94-451b-a985-20f7090f5c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:50.337824 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65e63c-9c94-451b-a985-20f7090f5c39 admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:50.338491 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc65e63c-9c94-451b-a985-20f7090f5c39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:50.340447 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc65e63c-9c94-451b-a985-20f7090f5c39 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:50.468024 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6263c1e1-19b7-4829-b839-37980f534c08 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:50.477437 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1169/2354] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:33:50 2022] POST /identity/v3/users => generated 286 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:33:50.480107 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6263c1e1-19b7-4829-b839-37980f534c08 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:50.485205 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7419207b-0c58-4f71-a669-814ca511aeaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:50.485935 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419207b-0c58-4f71-a669-814ca511aeaf admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:50.486317 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419207b-0c58-4f71-a669-814ca511aeaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:50.486690 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419207b-0c58-4f71-a669-814ca511aeaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:50.487502 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7419207b-0c58-4f71-a669-814ca511aeaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:50.489222 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7419207b-0c58-4f71-a669-814ca511aeaf admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:50.501004 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1186/2355] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:33:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:33:50.505320 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65e63c-9c94-451b-a985-20f7090f5c39 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:50.518787 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65e63c-9c94-451b-a985-20f7090f5c39 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:50.525997 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4cd4e-00b2-458c-900b-459748f74593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:50.526621 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4cd4e-00b2-458c-900b-459748f74593 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:50.526908 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4cd4e-00b2-458c-900b-459748f74593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:50.527186 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4cd4e-00b2-458c-900b-459748f74593 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:50.527887 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f4cd4e-00b2-458c-900b-459748f74593 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:50.529522 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f4cd4e-00b2-458c-900b-459748f74593 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:50.546287 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1170/2356] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:33:50 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:33:50.550392 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7419207b-0c58-4f71-a669-814ca511aeaf admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:50.568176 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7419207b-0c58-4f71-a669-814ca511aeaf admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:50.572925 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4422b13f-2096-4c69-96eb-50c0eef1bc82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:50.574083 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422b13f-2096-4c69-96eb-50c0eef1bc82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:50.574083 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422b13f-2096-4c69-96eb-50c0eef1bc82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:50.574289 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422b13f-2096-4c69-96eb-50c0eef1bc82 admin admin] PATH_INFO: `/v3/projects/d812c2a66f35492485c184641f509e17/users/4d812a76bcd5449ea4c4f17cf433633d/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:50.590141 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4422b13f-2096-4c69-96eb-50c0eef1bc82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d812c2a66f35492485c184641f509e17, user_id=4d812a76bcd5449ea4c4f17cf433633d, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:50.592873 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4422b13f-2096-4c69-96eb-50c0eef1bc82 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:50.595370 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:33:50.686325 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1187/2357] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:33:50 2022] PUT /identity/v3/projects/d812c2a66f35492485c184641f509e17/users/4d812a76bcd5449ea4c4f17cf433633d/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:33:50.697130 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1102a-5f77-4cd3-8746-4077005331e8 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:50.697419 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1102a-5f77-4cd3-8746-4077005331e8 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:50.697700 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1102a-5f77-4cd3-8746-4077005331e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:50.730351 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-30c1102a-5f77-4cd3-8746-4077005331e8 None None] MFA Rules not processed for user `4d812a76bcd5449ea4c4f17cf433633d`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:33:50.731578 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-30c1102a-5f77-4cd3-8746-4077005331e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:50.800229 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1171/2358] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:33:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:33:50.814075 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4422b13f-2096-4c69-96eb-50c0eef1bc82 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:50.815976 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-4422b13f-2096-4c69-96eb-50c0eef1bc82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:50.886216 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4422b13f-2096-4c69-96eb-50c0eef1bc82 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:50.890007 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d447764-e199-4df7-ada5-f2dcb34a26ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:50.890645 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d447764-e199-4df7-ada5-f2dcb34a26ed service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:50.890933 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d447764-e199-4df7-ada5-f2dcb34a26ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:50.891212 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d447764-e199-4df7-ada5-f2dcb34a26ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:50.900148 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d447764-e199-4df7-ada5-f2dcb34a26ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:50.901586 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d447764-e199-4df7-ada5-f2dcb34a26ed service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:50.910715 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1188/2359] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:33:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:33:50.935961 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c1102a-5f77-4cd3-8746-4077005331e8 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:50.998701 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c1102a-5f77-4cd3-8746-4077005331e8 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:51.003252 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dcd17f-d277-40f5-b2a7-27835f64c3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:51.003882 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dcd17f-d277-40f5-b2a7-27835f64c3c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:51.004203 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dcd17f-d277-40f5-b2a7-27835f64c3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:51.004488 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dcd17f-d277-40f5-b2a7-27835f64c3c1 admin admin] PATH_INFO: `/v3/users/4d812a76bcd5449ea4c4f17cf433633d` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:51.005660 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dcd17f-d277-40f5-b2a7-27835f64c3c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d812a76bcd5449ea4c4f17cf433633d)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:51.008111 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dcd17f-d277-40f5-b2a7-27835f64c3c1 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:51.030478 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:33:51.202495 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-f6dcd17f-d277-40f5-b2a7-27835f64c3c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d812a76bcd5449ea4c4f17cf433633d'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:51.209394 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-f6dcd17f-d277-40f5-b2a7-27835f64c3c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d812a76bcd5449ea4c4f17cf433633d'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:51.210061 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-f6dcd17f-d277-40f5-b2a7-27835f64c3c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d812a76bcd5449ea4c4f17cf433633d'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:51.221875 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-f6dcd17f-d277-40f5-b2a7-27835f64c3c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d812a76bcd5449ea4c4f17cf433633d'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:51.222549 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-f6dcd17f-d277-40f5-b2a7-27835f64c3c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d812a76bcd5449ea4c4f17cf433633d'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:51.308568 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1172/2360] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:33:50 2022] DELETE /identity/v3/users/4d812a76bcd5449ea4c4f17cf433633d => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:33:51.313115 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9006e-37c8-42d9-bc1b-8ccd36193db1 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:51.313408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9006e-37c8-42d9-bc1b-8ccd36193db1 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:51.313687 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9006e-37c8-42d9-bc1b-8ccd36193db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:51.335209 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-e1b9006e-37c8-42d9-bc1b-8ccd36193db1 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:33:51.336486 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-e1b9006e-37c8-42d9-bc1b-8ccd36193db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:51.397250 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1189/2361] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:33:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:33:51.407385 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dcd17f-d277-40f5-b2a7-27835f64c3c1 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:51.408912 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-f6dcd17f-d277-40f5-b2a7-27835f64c3c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:51.466657 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dcd17f-d277-40f5-b2a7-27835f64c3c1 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:51.470289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fdd753-d873-40fd-8b07-20405a2cc89e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:51.470905 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fdd753-d873-40fd-8b07-20405a2cc89e service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:51.471198 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fdd753-d873-40fd-8b07-20405a2cc89e service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:51.471471 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fdd753-d873-40fd-8b07-20405a2cc89e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:51.481140 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fdd753-d873-40fd-8b07-20405a2cc89e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:51.482571 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fdd753-d873-40fd-8b07-20405a2cc89e service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:51.492853 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1173/2362] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:33:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:33:51.947705 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b9006e-37c8-42d9-bc1b-8ccd36193db1 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:51.949597 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-e1b9006e-37c8-42d9-bc1b-8ccd36193db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:51.977912 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b9006e-37c8-42d9-bc1b-8ccd36193db1 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:51.987242 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd76f331-f110-4c3f-9c11-a61c5b9d54da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:51.988662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd76f331-f110-4c3f-9c11-a61c5b9d54da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:51.988984 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd76f331-f110-4c3f-9c11-a61c5b9d54da admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:51.989463 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd76f331-f110-4c3f-9c11-a61c5b9d54da admin admin] PATH_INFO: `/v3/projects/d812c2a66f35492485c184641f509e17` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:51.991011 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd76f331-f110-4c3f-9c11-a61c5b9d54da admin admin] RBAC: Authorizing `identity:delete_project(project_id=d812c2a66f35492485c184641f509e17)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:51.993424 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd76f331-f110-4c3f-9c11-a61c5b9d54da admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:52.126275 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-cd76f331-f110-4c3f-9c11-a61c5b9d54da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd812c2a66f35492485c184641f509e17', 'request_id': 'req-cd76f331-f110-4c3f-9c11-a61c5b9d54da'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:52.132772 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-cd76f331-f110-4c3f-9c11-a61c5b9d54da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd812c2a66f35492485c184641f509e17', 'request_id': 'req-cd76f331-f110-4c3f-9c11-a61c5b9d54da'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:52.140224 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-cd76f331-f110-4c3f-9c11-a61c5b9d54da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd812c2a66f35492485c184641f509e17', 'request_id': 'req-cd76f331-f110-4c3f-9c11-a61c5b9d54da'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:52.210915 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-cd76f331-f110-4c3f-9c11-a61c5b9d54da admin admin] The token cache is being invalidate because project d812c2a66f35492485c184641f509e17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:33:52.211332 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-cd76f331-f110-4c3f-9c11-a61c5b9d54da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:52.212004 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-cd76f331-f110-4c3f-9c11-a61c5b9d54da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:33:52.213551 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1190/2363] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:33:51 2022] DELETE /identity/v3/projects/d812c2a66f35492485c184641f509e17 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:33:54.149429 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8765c0-4fde-4eba-a56b-534b6f9d77a7 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:54.149899 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8765c0-4fde-4eba-a56b-534b6f9d77a7 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:54.149992 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8765c0-4fde-4eba-a56b-534b6f9d77a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:54.172304 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-8f8765c0-4fde-4eba-a56b-534b6f9d77a7 None None] MFA Rules not processed for user `1654fb6a81e14d6b99ab495bb6d029d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:33:54.172304 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-8f8765c0-4fde-4eba-a56b-534b6f9d77a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:54.231633 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1174/2364] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 5 15:33:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:33:54.245853 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd76f331-f110-4c3f-9c11-a61c5b9d54da admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:33:54.247516 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-cd76f331-f110-4c3f-9c11-a61c5b9d54da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:33:54.272052 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:33:54.321156 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd76f331-f110-4c3f-9c11-a61c5b9d54da admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:33:54.326802 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59124b8-dc16-427c-82d7-45b2a916bc46 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85154ce7446245738edcc8702dbf6861', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:33:54.327481 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59124b8-dc16-427c-82d7-45b2a916bc46 service swift] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:33:54.327762 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59124b8-dc16-427c-82d7-45b2a916bc46 service swift] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:33:54.328040 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59124b8-dc16-427c-82d7-45b2a916bc46 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:33:54.336522 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59124b8-dc16-427c-82d7-45b2a916bc46 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:33:54.338237 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59124b8-dc16-427c-82d7-45b2a916bc46 service swift] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:33:54.346392 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1191/2365] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 5 15:33:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 15:34:04.223282 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:DELETE=7,keystone:SELECT=71,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:34:04.348366 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=23 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:34:05.308244 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8765c0-4fde-4eba-a56b-534b6f9d77a7 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:34:05.309875 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-8f8765c0-4fde-4eba-a56b-534b6f9d77a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:34:05.374110 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8765c0-4fde-4eba-a56b-534b6f9d77a7 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:34:05.378097 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6111cd-1a56-4569-a3d7-953a756b6703 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:34:05.378726 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6111cd-1a56-4569-a3d7-953a756b6703 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:34:05.379048 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6111cd-1a56-4569-a3d7-953a756b6703 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:34:05.379492 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6111cd-1a56-4569-a3d7-953a756b6703 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:34:05.382112 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-6e6111cd-1a56-4569-a3d7-953a756b6703 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:34:05.453328 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6111cd-1a56-4569-a3d7-953a756b6703 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:34:05.454775 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6111cd-1a56-4569-a3d7-953a756b6703 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:34:05.465465 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1175/2366] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:34:05 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:34:06.362038 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59124b8-dc16-427c-82d7-45b2a916bc46 service swift] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:34:06.363471 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-c59124b8-dc16-427c-82d7-45b2a916bc46 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:34:06.439889 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59124b8-dc16-427c-82d7-45b2a916bc46 service swift] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:34:06.446959 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3367c7a8-88c3-4333-a60d-e802520446cd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '107a7c18a2c6440b86e1b7c3e39cc724', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:34:06.447847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367c7a8-88c3-4333-a60d-e802520446cd service placement] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:34:06.448170 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367c7a8-88c3-4333-a60d-e802520446cd service placement] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:34:06.448452 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367c7a8-88c3-4333-a60d-e802520446cd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:34:06.450273 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-3367c7a8-88c3-4333-a60d-e802520446cd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:34:06.462860 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3367c7a8-88c3-4333-a60d-e802520446cd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:34:06.465574 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3367c7a8-88c3-4333-a60d-e802520446cd service placement] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:34:06.478709 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1192/2367] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:34:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:34:15.460417 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=50 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:34:16.473509 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=32 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:34:17.664823 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a41ea4-2b39-44e0-b4d1-90c9499fff2c None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:34:17.665387 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a41ea4-2b39-44e0-b4d1-90c9499fff2c None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:34:17.665387 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a41ea4-2b39-44e0-b4d1-90c9499fff2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:34:17.684751 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-54a41ea4-2b39-44e0-b4d1-90c9499fff2c None None] MFA Rules not processed for user `1654fb6a81e14d6b99ab495bb6d029d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:34:17.685990 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-54a41ea4-2b39-44e0-b4d1-90c9499fff2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:34:17.693609 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1176/2368] 10.222.0.40 () {58 vars in 967 bytes} [Fri Aug 5 15:34:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:34:17.706727 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3367c7a8-88c3-4333-a60d-e802520446cd service placement] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:34:17.718355 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3367c7a8-88c3-4333-a60d-e802520446cd service placement] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:34:17.723406 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741db4f6-ee94-41bf-85f4-7063ac630c8c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85154ce7446245738edcc8702dbf6861', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:34:17.724012 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741db4f6-ee94-41bf-85f4-7063ac630c8c service swift] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:34:17.724336 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741db4f6-ee94-41bf-85f4-7063ac630c8c service swift] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:34:17.724617 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741db4f6-ee94-41bf-85f4-7063ac630c8c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:34:17.733239 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741db4f6-ee94-41bf-85f4-7063ac630c8c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:34:17.734855 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741db4f6-ee94-41bf-85f4-7063ac630c8c service swift] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:34:17.742837 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1193/2369] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 5 15:34:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 15:34:27.678755 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=3 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:34:27.739681 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=6 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:34:33.903620 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161acc8-a462-4b92-b831-02bc27b381f7 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:34:33.904139 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161acc8-a462-4b92-b831-02bc27b381f7 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:34:33.904235 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161acc8-a462-4b92-b831-02bc27b381f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:34:33.923907 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-4161acc8-a462-4b92-b831-02bc27b381f7 None None] MFA Rules not processed for user `cb055110c77543c1861dfd8c155d7582`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:34:33.925167 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-4161acc8-a462-4b92-b831-02bc27b381f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:34:33.980859 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1177/2370] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:34:33 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:34:43.973828 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=25 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:35:07.490573 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e804c-a590-47a2-95c6-fc7883d31b58 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:07.491086 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e804c-a590-47a2-95c6-fc7883d31b58 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:07.491205 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e804c-a590-47a2-95c6-fc7883d31b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:07.510753 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-3c5e804c-a590-47a2-95c6-fc7883d31b58 None None] MFA Rules not processed for user `d36b38a6e2d34c41b827ff6db9742332`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:35:07.514994 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-3c5e804c-a590-47a2-95c6-fc7883d31b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:35:07.566632 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1194/2371] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:35:07 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:35:16.073998 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd12cd-5c16-4af1-9d27-4ffdde934129 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:16.074547 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd12cd-5c16-4af1-9d27-4ffdde934129 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:16.074547 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd12cd-5c16-4af1-9d27-4ffdde934129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:16.093167 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-1abd12cd-5c16-4af1-9d27-4ffdde934129 None None] MFA Rules not processed for user `04e80222936d4427909414a78e1dc489`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:35:16.094366 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-1abd12cd-5c16-4af1-9d27-4ffdde934129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:35:16.148130 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1178/2372] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:35:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:35:17.559844 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=25 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:35:26.141131 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=25 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:35:45.979868 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e804c-a590-47a2-95c6-fc7883d31b58 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:45.985025 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-3c5e804c-a590-47a2-95c6-fc7883d31b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:35:46.047028 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e804c-a590-47a2-95c6-fc7883d31b58 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:35:46.051524 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4404839-3bcb-42bf-87e4-a76e3b5bcf73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:35:46.052199 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4404839-3bcb-42bf-87e4-a76e3b5bcf73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:46.052486 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4404839-3bcb-42bf-87e4-a76e3b5bcf73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:46.052769 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4404839-3bcb-42bf-87e4-a76e3b5bcf73 admin admin] PATH_INFO: `/v3/users/cb055110c77543c1861dfd8c155d7582` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:46.053940 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4404839-3bcb-42bf-87e4-a76e3b5bcf73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb055110c77543c1861dfd8c155d7582)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:35:46.055860 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4404839-3bcb-42bf-87e4-a76e3b5bcf73 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:35:46.276566 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-a4404839-3bcb-42bf-87e4-a76e3b5bcf73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb055110c77543c1861dfd8c155d7582'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:35:46.277269 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-a4404839-3bcb-42bf-87e4-a76e3b5bcf73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb055110c77543c1861dfd8c155d7582'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:35:46.289405 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-a4404839-3bcb-42bf-87e4-a76e3b5bcf73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb055110c77543c1861dfd8c155d7582'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:35:46.377186 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-a4404839-3bcb-42bf-87e4-a76e3b5bcf73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb055110c77543c1861dfd8c155d7582'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:35:46.384839 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-a4404839-3bcb-42bf-87e4-a76e3b5bcf73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb055110c77543c1861dfd8c155d7582'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:35:46.391265 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1195/2373] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:35:45 2022] DELETE /identity/v3/users/cb055110c77543c1861dfd8c155d7582 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:35:46.401124 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd12cd-5c16-4af1-9d27-4ffdde934129 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:46.402549 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-1abd12cd-5c16-4af1-9d27-4ffdde934129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:35:46.414615 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd12cd-5c16-4af1-9d27-4ffdde934129 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:35:46.464323 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd364a06-18af-49be-bdda-35418b3ae202 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:35:46.464943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd364a06-18af-49be-bdda-35418b3ae202 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:46.465227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd364a06-18af-49be-bdda-35418b3ae202 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:46.465504 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd364a06-18af-49be-bdda-35418b3ae202 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:46.467246 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-cd364a06-18af-49be-bdda-35418b3ae202 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:35:46.529442 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd364a06-18af-49be-bdda-35418b3ae202 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:35:46.530945 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd364a06-18af-49be-bdda-35418b3ae202 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:35:46.540648 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1179/2374] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:35:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:35:46.795783 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4404839-3bcb-42bf-87e4-a76e3b5bcf73 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:46.797255 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-a4404839-3bcb-42bf-87e4-a76e3b5bcf73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:35:46.811469 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4404839-3bcb-42bf-87e4-a76e3b5bcf73 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:35:46.816171 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f515a4f2-36a0-4845-a103-0f1e240cb962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:35:46.816838 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515a4f2-36a0-4845-a103-0f1e240cb962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:46.817121 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515a4f2-36a0-4845-a103-0f1e240cb962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:46.817398 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515a4f2-36a0-4845-a103-0f1e240cb962 admin admin] PATH_INFO: `/v3/projects/1cb48dbd26c747d290bf3e666e65027a` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:46.818579 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f515a4f2-36a0-4845-a103-0f1e240cb962 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cb48dbd26c747d290bf3e666e65027a)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:35:46.821589 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f515a4f2-36a0-4845-a103-0f1e240cb962 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:35:46.920373 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-f515a4f2-36a0-4845-a103-0f1e240cb962 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cb48dbd26c747d290bf3e666e65027a', 'request_id': 'req-f515a4f2-36a0-4845-a103-0f1e240cb962'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:35:46.927175 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-f515a4f2-36a0-4845-a103-0f1e240cb962 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cb48dbd26c747d290bf3e666e65027a', 'request_id': 'req-f515a4f2-36a0-4845-a103-0f1e240cb962'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:35:46.932889 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-f515a4f2-36a0-4845-a103-0f1e240cb962 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cb48dbd26c747d290bf3e666e65027a', 'request_id': 'req-f515a4f2-36a0-4845-a103-0f1e240cb962'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:35:47.016981 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-f515a4f2-36a0-4845-a103-0f1e240cb962 admin admin] The token cache is being invalidate because project 1cb48dbd26c747d290bf3e666e65027a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:35:47.017581 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-f515a4f2-36a0-4845-a103-0f1e240cb962 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:35:47.017956 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-f515a4f2-36a0-4845-a103-0f1e240cb962 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:35:47.019475 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1196/2375] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:35:46 2022] DELETE /identity/v3/projects/1cb48dbd26c747d290bf3e666e65027a => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:35:47.024474 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06707d-0c4e-4786-9b88-b95c81965968 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:47.024772 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06707d-0c4e-4786-9b88-b95c81965968 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:47.025057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06707d-0c4e-4786-9b88-b95c81965968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:47.045267 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-db06707d-0c4e-4786-9b88-b95c81965968 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:35:47.046398 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-db06707d-0c4e-4786-9b88-b95c81965968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:35:47.104312 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1180/2376] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:35:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:35:47.111859 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5197a-f5d0-4206-84a2-387cb3714177 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:47.112180 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5197a-f5d0-4206-84a2-387cb3714177 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:47.112460 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5197a-f5d0-4206-84a2-387cb3714177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:47.120009 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:35:47.131976 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-9be5197a-f5d0-4206-84a2-387cb3714177 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:35:47.133205 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-9be5197a-f5d0-4206-84a2-387cb3714177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:35:47.141822 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1197/2377] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:35:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:35:47.145928 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db06707d-0c4e-4786-9b88-b95c81965968 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:47.155688 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db06707d-0c4e-4786-9b88-b95c81965968 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:35:47.160900 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c1aa9-31cf-419f-84fe-3b251f53461e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:35:47.161564 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c1aa9-31cf-419f-84fe-3b251f53461e admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:47.161851 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c1aa9-31cf-419f-84fe-3b251f53461e admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:47.162138 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c1aa9-31cf-419f-84fe-3b251f53461e admin admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:47.162832 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6c1aa9-31cf-419f-84fe-3b251f53461e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:35:47.163144 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6c1aa9-31cf-419f-84fe-3b251f53461e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:35:47.164844 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6c1aa9-31cf-419f-84fe-3b251f53461e admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:35:47.176679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1181/2378] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:35:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:35:47.181100 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5197a-f5d0-4206-84a2-387cb3714177 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:47.191016 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5197a-f5d0-4206-84a2-387cb3714177 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:35:47.195643 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a8398-c59c-4b7e-80c8-c39891166833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:35:47.196285 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a8398-c59c-4b7e-80c8-c39891166833 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:47.196566 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a8398-c59c-4b7e-80c8-c39891166833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:47.196840 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a8398-c59c-4b7e-80c8-c39891166833 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:47.197507 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81a8398-c59c-4b7e-80c8-c39891166833 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:35:47.199426 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81a8398-c59c-4b7e-80c8-c39891166833 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:35:47.281030 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1198/2379] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:35:47 2022] POST /identity/v3/projects => generated 371 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:35:47.285729 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c1aa9-31cf-419f-84fe-3b251f53461e admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:47.295746 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c1aa9-31cf-419f-84fe-3b251f53461e admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:35:47.328422 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:35:47.351975 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbc048-c3df-424c-913b-2937c15c85a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:35:47.353638 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbc048-c3df-424c-913b-2937c15c85a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:47.353922 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbc048-c3df-424c-913b-2937c15c85a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:47.354195 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbc048-c3df-424c-913b-2937c15c85a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:47.354870 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbbc048-c3df-424c-913b-2937c15c85a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:35:47.356805 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbbc048-c3df-424c-913b-2937c15c85a0 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:35:47.435438 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1182/2380] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:35:47 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:35:47.439886 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a8398-c59c-4b7e-80c8-c39891166833 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:47.450889 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a8398-c59c-4b7e-80c8-c39891166833 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:35:47.459380 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb5b93-dde3-4c17-984d-35cc0031b8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:35:47.460018 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb5b93-dde3-4c17-984d-35cc0031b8cd admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:47.460338 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb5b93-dde3-4c17-984d-35cc0031b8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:47.460620 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb5b93-dde3-4c17-984d-35cc0031b8cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:47.461378 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb5b93-dde3-4c17-984d-35cc0031b8cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:35:47.463023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb5b93-dde3-4c17-984d-35cc0031b8cd admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:35:47.473411 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1199/2381] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:35:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:35:47.477414 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbc048-c3df-424c-913b-2937c15c85a0 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:47.487961 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbc048-c3df-424c-913b-2937c15c85a0 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:35:47.492917 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1087451-fe1e-48f6-9e20-8cf9210567ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:35:47.497666 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1087451-fe1e-48f6-9e20-8cf9210567ac admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:47.497666 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1087451-fe1e-48f6-9e20-8cf9210567ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:47.497666 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1087451-fe1e-48f6-9e20-8cf9210567ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:47.497666 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1087451-fe1e-48f6-9e20-8cf9210567ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:35:47.497666 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1087451-fe1e-48f6-9e20-8cf9210567ac admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:35:47.507202 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1183/2382] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:35:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:35:47.511784 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb5b93-dde3-4c17-984d-35cc0031b8cd admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:47.522700 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb5b93-dde3-4c17-984d-35cc0031b8cd admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:35:47.527403 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8cc09a-5342-4290-aafa-746602fedd21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:35:47.528093 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8cc09a-5342-4290-aafa-746602fedd21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:47.528389 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8cc09a-5342-4290-aafa-746602fedd21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:47.528673 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8cc09a-5342-4290-aafa-746602fedd21 admin admin] PATH_INFO: `/v3/projects/2efa0c365169481995ab394c9ec0f815/users/1507bf6fa3154cab9213580b28c0b57a/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:47.543616 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8cc09a-5342-4290-aafa-746602fedd21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2efa0c365169481995ab394c9ec0f815, user_id=1507bf6fa3154cab9213580b28c0b57a, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:35:47.549322 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8cc09a-5342-4290-aafa-746602fedd21 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:35:47.605432 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1200/2383] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:35:47 2022] PUT /identity/v3/projects/2efa0c365169481995ab394c9ec0f815/users/1507bf6fa3154cab9213580b28c0b57a/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:35:47.616082 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ab496-abfc-4bd5-8dc1-64143e459c86 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:47.616420 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ab496-abfc-4bd5-8dc1-64143e459c86 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:47.616747 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ab496-abfc-4bd5-8dc1-64143e459c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:47.644758 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-226ab496-abfc-4bd5-8dc1-64143e459c86 None None] MFA Rules not processed for user `1507bf6fa3154cab9213580b28c0b57a`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:35:47.645978 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-226ab496-abfc-4bd5-8dc1-64143e459c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:35:47.709308 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1184/2384] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:35:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:35:47.720714 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8cc09a-5342-4290-aafa-746602fedd21 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:47.722408 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-6d8cc09a-5342-4290-aafa-746602fedd21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:35:47.794501 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8cc09a-5342-4290-aafa-746602fedd21 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:35:47.800218 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43183e-9e45-4a05-bf0b-ef8b126901d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:35:47.800771 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43183e-9e45-4a05-bf0b-ef8b126901d8 service nova] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:47.801060 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43183e-9e45-4a05-bf0b-ef8b126901d8 service nova] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:47.801341 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43183e-9e45-4a05-bf0b-ef8b126901d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:47.809060 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c43183e-9e45-4a05-bf0b-ef8b126901d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:35:47.810638 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c43183e-9e45-4a05-bf0b-ef8b126901d8 service nova] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:35:47.819420 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1201/2385] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:35:47 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:35:48.412664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ab496-abfc-4bd5-8dc1-64143e459c86 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:48.414033 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-226ab496-abfc-4bd5-8dc1-64143e459c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:35:48.472823 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ab496-abfc-4bd5-8dc1-64143e459c86 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:35:48.476480 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2332d-e8bb-4bde-a755-a0fdc53bf620 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:35:48.477098 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2332d-e8bb-4bde-a755-a0fdc53bf620 service cinder] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:48.477380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2332d-e8bb-4bde-a755-a0fdc53bf620 service cinder] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:48.477659 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2332d-e8bb-4bde-a755-a0fdc53bf620 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:48.479425 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-31d2332d-e8bb-4bde-a755-a0fdc53bf620 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:35:48.493199 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d2332d-e8bb-4bde-a755-a0fdc53bf620 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:35:48.494620 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d2332d-e8bb-4bde-a755-a0fdc53bf620 service cinder] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:35:48.506188 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1185/2386] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:35:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:35:49.573970 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43183e-9e45-4a05-bf0b-ef8b126901d8 service nova] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:49.575391 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-0c43183e-9e45-4a05-bf0b-ef8b126901d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:35:49.587670 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43183e-9e45-4a05-bf0b-ef8b126901d8 service nova] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:35:49.591542 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c60516-9b41-4a85-a6ec-731455a4c07f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:35:49.592186 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c60516-9b41-4a85-a6ec-731455a4c07f service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:49.592480 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c60516-9b41-4a85-a6ec-731455a4c07f service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:49.592765 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c60516-9b41-4a85-a6ec-731455a4c07f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:49.595131 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-12c60516-9b41-4a85-a6ec-731455a4c07f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:35:49.608348 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c60516-9b41-4a85-a6ec-731455a4c07f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:35:49.609781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c60516-9b41-4a85-a6ec-731455a4c07f service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:35:49.620672 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1202/2387] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:35:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:35:51.004450 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2332d-e8bb-4bde-a755-a0fdc53bf620 service cinder] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:51.052329 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:35:51.066314 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2332d-e8bb-4bde-a755-a0fdc53bf620 service cinder] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:35:51.070947 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e24d94-abe0-4cb1-98be-99290415582a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:35:51.071583 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e24d94-abe0-4cb1-98be-99290415582a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:51.071872 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e24d94-abe0-4cb1-98be-99290415582a admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:51.072184 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e24d94-abe0-4cb1-98be-99290415582a admin admin] PATH_INFO: `/v3/users/1507bf6fa3154cab9213580b28c0b57a` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:51.073356 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e24d94-abe0-4cb1-98be-99290415582a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1507bf6fa3154cab9213580b28c0b57a)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:35:51.075274 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e24d94-abe0-4cb1-98be-99290415582a admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:35:51.221153 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-06e24d94-abe0-4cb1-98be-99290415582a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1507bf6fa3154cab9213580b28c0b57a'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:35:51.228080 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-06e24d94-abe0-4cb1-98be-99290415582a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1507bf6fa3154cab9213580b28c0b57a'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:35:51.228763 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-06e24d94-abe0-4cb1-98be-99290415582a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1507bf6fa3154cab9213580b28c0b57a'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:35:51.241183 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-06e24d94-abe0-4cb1-98be-99290415582a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1507bf6fa3154cab9213580b28c0b57a'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:35:51.241810 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-06e24d94-abe0-4cb1-98be-99290415582a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1507bf6fa3154cab9213580b28c0b57a'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:35:51.296532 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1186/2388] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:35:51 2022] DELETE /identity/v3/users/1507bf6fa3154cab9213580b28c0b57a => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:35:51.301653 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14167d5f-386b-48e9-928f-ee908cfeb02b None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:51.302129 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14167d5f-386b-48e9-928f-ee908cfeb02b None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:51.302226 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14167d5f-386b-48e9-928f-ee908cfeb02b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:51.321152 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-14167d5f-386b-48e9-928f-ee908cfeb02b None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:35:51.322375 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-14167d5f-386b-48e9-928f-ee908cfeb02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:35:51.382877 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1203/2389] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:35:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:35:51.392976 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e24d94-abe0-4cb1-98be-99290415582a admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:51.394393 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-06e24d94-abe0-4cb1-98be-99290415582a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:35:51.451857 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e24d94-abe0-4cb1-98be-99290415582a admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:35:51.456087 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838179c-012c-47bf-8d0d-39df40bca1f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:35:51.456711 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838179c-012c-47bf-8d0d-39df40bca1f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:51.456998 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838179c-012c-47bf-8d0d-39df40bca1f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:51.457278 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838179c-012c-47bf-8d0d-39df40bca1f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:51.466274 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c838179c-012c-47bf-8d0d-39df40bca1f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:35:51.467704 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c838179c-012c-47bf-8d0d-39df40bca1f5 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:35:51.478079 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1187/2390] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:35:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:35:51.876808 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14167d5f-386b-48e9-928f-ee908cfeb02b None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:51.878374 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-14167d5f-386b-48e9-928f-ee908cfeb02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:35:51.892278 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14167d5f-386b-48e9-928f-ee908cfeb02b None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:35:51.896997 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f91d06a-a490-4da7-8b83-fa12d16d4708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:35:51.897659 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91d06a-a490-4da7-8b83-fa12d16d4708 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:51.897965 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91d06a-a490-4da7-8b83-fa12d16d4708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:51.898248 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91d06a-a490-4da7-8b83-fa12d16d4708 admin admin] PATH_INFO: `/v3/projects/2efa0c365169481995ab394c9ec0f815` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:51.899422 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f91d06a-a490-4da7-8b83-fa12d16d4708 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2efa0c365169481995ab394c9ec0f815)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:35:51.901403 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f91d06a-a490-4da7-8b83-fa12d16d4708 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:35:51.908297 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:35:52.001807 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-7f91d06a-a490-4da7-8b83-fa12d16d4708 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2efa0c365169481995ab394c9ec0f815', 'request_id': 'req-7f91d06a-a490-4da7-8b83-fa12d16d4708'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:35:52.007872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-7f91d06a-a490-4da7-8b83-fa12d16d4708 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2efa0c365169481995ab394c9ec0f815', 'request_id': 'req-7f91d06a-a490-4da7-8b83-fa12d16d4708'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:35:52.014062 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-7f91d06a-a490-4da7-8b83-fa12d16d4708 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2efa0c365169481995ab394c9ec0f815', 'request_id': 'req-7f91d06a-a490-4da7-8b83-fa12d16d4708'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:35:52.074006 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-7f91d06a-a490-4da7-8b83-fa12d16d4708 admin admin] The token cache is being invalidate because project 2efa0c365169481995ab394c9ec0f815 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:35:52.074297 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-7f91d06a-a490-4da7-8b83-fa12d16d4708 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:35:52.074930 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-7f91d06a-a490-4da7-8b83-fa12d16d4708 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:35:52.076475 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1204/2391] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:35:51 2022] DELETE /identity/v3/projects/2efa0c365169481995ab394c9ec0f815 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:35:52.082081 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c7a04-5c46-4987-b87b-dbdab9519f40 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:52.082565 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c7a04-5c46-4987-b87b-dbdab9519f40 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:52.082670 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c7a04-5c46-4987-b87b-dbdab9519f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:52.102170 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-b81c7a04-5c46-4987-b87b-dbdab9519f40 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:35:52.103368 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-b81c7a04-5c46-4987-b87b-dbdab9519f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:35:52.161396 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1188/2392] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:35:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:35:52.170856 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbc93f-8769-4887-bc67-9d99c9b727ff None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:52.171150 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbc93f-8769-4887-bc67-9d99c9b727ff None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:52.171458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbc93f-8769-4887-bc67-9d99c9b727ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:52.191439 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-01bbc93f-8769-4887-bc67-9d99c9b727ff None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:35:52.192673 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-01bbc93f-8769-4887-bc67-9d99c9b727ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:35:52.200530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1205/2393] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:35:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:35:52.205280 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81c7a04-5c46-4987-b87b-dbdab9519f40 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:52.215081 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81c7a04-5c46-4987-b87b-dbdab9519f40 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:35:52.219715 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d16e172-cce2-4579-9598-f2946993cde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:35:52.220375 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16e172-cce2-4579-9598-f2946993cde8 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:52.220663 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16e172-cce2-4579-9598-f2946993cde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:52.220941 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16e172-cce2-4579-9598-f2946993cde8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:52.221632 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d16e172-cce2-4579-9598-f2946993cde8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:35:52.221942 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d16e172-cce2-4579-9598-f2946993cde8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:35:52.223528 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d16e172-cce2-4579-9598-f2946993cde8 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:35:52.230216 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:35:52.235858 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1189/2394] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:35:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:35:52.239976 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bbc93f-8769-4887-bc67-9d99c9b727ff None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:52.249149 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bbc93f-8769-4887-bc67-9d99c9b727ff None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:35:52.263372 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2026491e-325e-4dc2-bf6f-c2b897b87450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:35:52.263372 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026491e-325e-4dc2-bf6f-c2b897b87450 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:52.263372 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026491e-325e-4dc2-bf6f-c2b897b87450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:52.263372 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026491e-325e-4dc2-bf6f-c2b897b87450 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:52.263751 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2026491e-325e-4dc2-bf6f-c2b897b87450 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:35:52.263751 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2026491e-325e-4dc2-bf6f-c2b897b87450 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:35:52.322179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1206/2395] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:35:52 2022] POST /identity/v3/projects => generated 375 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:35:52.326626 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d16e172-cce2-4579-9598-f2946993cde8 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:52.336600 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d16e172-cce2-4579-9598-f2946993cde8 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:35:52.393531 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c0f5a-00ce-43a0-ab79-78d4ead462d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:35:52.394146 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c0f5a-00ce-43a0-ab79-78d4ead462d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:52.394433 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c0f5a-00ce-43a0-ab79-78d4ead462d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:52.394741 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c0f5a-00ce-43a0-ab79-78d4ead462d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:52.395410 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5c0f5a-00ce-43a0-ab79-78d4ead462d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:35:52.397417 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5c0f5a-00ce-43a0-ab79-78d4ead462d0 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:35:52.463686 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1190/2396] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:35:52 2022] POST /identity/v3/users => generated 288 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:35:52.467821 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2026491e-325e-4dc2-bf6f-c2b897b87450 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:52.478458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2026491e-325e-4dc2-bf6f-c2b897b87450 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:35:52.483036 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99dfd64-fa37-4e2a-b698-9bfbd0ca05af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:35:52.483641 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99dfd64-fa37-4e2a-b698-9bfbd0ca05af admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:52.483924 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99dfd64-fa37-4e2a-b698-9bfbd0ca05af admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:52.484237 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99dfd64-fa37-4e2a-b698-9bfbd0ca05af admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:52.484927 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99dfd64-fa37-4e2a-b698-9bfbd0ca05af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:35:52.487090 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99dfd64-fa37-4e2a-b698-9bfbd0ca05af admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:35:52.496899 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1207/2397] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:35:52.501550 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c0f5a-00ce-43a0-ab79-78d4ead462d0 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:52.511613 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c0f5a-00ce-43a0-ab79-78d4ead462d0 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:35:52.516633 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335aa374-ebf2-49a0-8d5b-21c0489edb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:35:52.517157 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335aa374-ebf2-49a0-8d5b-21c0489edb0f admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:52.517445 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335aa374-ebf2-49a0-8d5b-21c0489edb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:52.517719 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335aa374-ebf2-49a0-8d5b-21c0489edb0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:52.518439 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335aa374-ebf2-49a0-8d5b-21c0489edb0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:35:52.520054 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335aa374-ebf2-49a0-8d5b-21c0489edb0f admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:35:52.530484 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1191/2398] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:35:52.534518 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99dfd64-fa37-4e2a-b698-9bfbd0ca05af admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:52.544395 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99dfd64-fa37-4e2a-b698-9bfbd0ca05af admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:35:52.549574 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535fb6d-4021-4d34-b9ab-f3414b0691dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:35:52.550222 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535fb6d-4021-4d34-b9ab-f3414b0691dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:52.550508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535fb6d-4021-4d34-b9ab-f3414b0691dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:52.550787 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535fb6d-4021-4d34-b9ab-f3414b0691dd admin admin] PATH_INFO: `/v3/projects/7f02796bb02749f8b1a28505148f07b8/users/a4eff17edc5a4cc29fe2cf81e0452443/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:52.565941 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c535fb6d-4021-4d34-b9ab-f3414b0691dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f02796bb02749f8b1a28505148f07b8, user_id=a4eff17edc5a4cc29fe2cf81e0452443, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:35:52.568634 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c535fb6d-4021-4d34-b9ab-f3414b0691dd admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:35:52.646184 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1208/2399] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:35:52 2022] PUT /identity/v3/projects/7f02796bb02749f8b1a28505148f07b8/users/a4eff17edc5a4cc29fe2cf81e0452443/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:35:52.656785 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3938386-6a66-413d-b9dc-2901b6fbd565 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:52.657589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3938386-6a66-413d-b9dc-2901b6fbd565 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:52.657894 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3938386-6a66-413d-b9dc-2901b6fbd565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:52.686141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-b3938386-6a66-413d-b9dc-2901b6fbd565 None None] MFA Rules not processed for user `a4eff17edc5a4cc29fe2cf81e0452443`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:35:52.687369 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-b3938386-6a66-413d-b9dc-2901b6fbd565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:35:52.753057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1192/2400] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:35:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:35:52.763986 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535fb6d-4021-4d34-b9ab-f3414b0691dd admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:52.765729 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-c535fb6d-4021-4d34-b9ab-f3414b0691dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:35:52.829970 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535fb6d-4021-4d34-b9ab-f3414b0691dd admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:35:52.834750 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65489af-dc06-46df-988c-9ce351c27f05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:35:52.838625 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65489af-dc06-46df-988c-9ce351c27f05 service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:52.838625 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65489af-dc06-46df-988c-9ce351c27f05 service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:52.838625 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65489af-dc06-46df-988c-9ce351c27f05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:52.855073 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65489af-dc06-46df-988c-9ce351c27f05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:35:52.861386 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65489af-dc06-46df-988c-9ce351c27f05 service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:35:52.871289 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1209/2401] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:35:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:35:56.978572 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3938386-6a66-413d-b9dc-2901b6fbd565 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:57.040373 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3938386-6a66-413d-b9dc-2901b6fbd565 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:35:57.045106 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15260a2f-17f0-4a27-9f90-d5068de22bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:35:57.045748 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15260a2f-17f0-4a27-9f90-d5068de22bce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:57.046033 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15260a2f-17f0-4a27-9f90-d5068de22bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:57.046313 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15260a2f-17f0-4a27-9f90-d5068de22bce admin admin] PATH_INFO: `/v3/users/a4eff17edc5a4cc29fe2cf81e0452443` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:57.047472 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15260a2f-17f0-4a27-9f90-d5068de22bce admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4eff17edc5a4cc29fe2cf81e0452443)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:35:57.049960 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15260a2f-17f0-4a27-9f90-d5068de22bce admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:35:57.055181 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:35:57.214538 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-15260a2f-17f0-4a27-9f90-d5068de22bce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4eff17edc5a4cc29fe2cf81e0452443'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:35:57.222035 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-15260a2f-17f0-4a27-9f90-d5068de22bce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4eff17edc5a4cc29fe2cf81e0452443'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:35:57.222710 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-15260a2f-17f0-4a27-9f90-d5068de22bce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4eff17edc5a4cc29fe2cf81e0452443'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:35:57.234518 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-15260a2f-17f0-4a27-9f90-d5068de22bce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4eff17edc5a4cc29fe2cf81e0452443'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:35:57.235158 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-15260a2f-17f0-4a27-9f90-d5068de22bce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4eff17edc5a4cc29fe2cf81e0452443'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:35:57.305013 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1193/2402] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:35:56 2022] DELETE /identity/v3/users/a4eff17edc5a4cc29fe2cf81e0452443 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:35:57.314178 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a1994e-7e4b-4f1b-b3d2-14d5e515f370 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:57.314697 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a1994e-7e4b-4f1b-b3d2-14d5e515f370 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:57.314980 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a1994e-7e4b-4f1b-b3d2-14d5e515f370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:57.334522 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-82a1994e-7e4b-4f1b-b3d2-14d5e515f370 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:35:57.335739 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-82a1994e-7e4b-4f1b-b3d2-14d5e515f370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:35:57.366913 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:35:57.398603 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1210/2403] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:35:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:35:57.410112 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15260a2f-17f0-4a27-9f90-d5068de22bce admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:57.411569 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-15260a2f-17f0-4a27-9f90-d5068de22bce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:35:57.475799 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15260a2f-17f0-4a27-9f90-d5068de22bce admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:35:57.479715 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b68b41-8cd5-401a-985c-fd4826c3e97f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:35:57.480292 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b68b41-8cd5-401a-985c-fd4826c3e97f service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:57.480575 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b68b41-8cd5-401a-985c-fd4826c3e97f service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:57.480857 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b68b41-8cd5-401a-985c-fd4826c3e97f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:57.490331 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b68b41-8cd5-401a-985c-fd4826c3e97f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:35:57.491825 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b68b41-8cd5-401a-985c-fd4826c3e97f service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:35:57.501416 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1194/2404] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:35:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 15:35:57.859769 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a1994e-7e4b-4f1b-b3d2-14d5e515f370 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:57.861280 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-82a1994e-7e4b-4f1b-b3d2-14d5e515f370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:35:57.875736 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a1994e-7e4b-4f1b-b3d2-14d5e515f370 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:35:57.881623 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42184de-512e-4423-a2eb-8f5f633cf22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:35:57.884848 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42184de-512e-4423-a2eb-8f5f633cf22e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:57.884848 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42184de-512e-4423-a2eb-8f5f633cf22e admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:57.884848 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42184de-512e-4423-a2eb-8f5f633cf22e admin admin] PATH_INFO: `/v3/projects/7f02796bb02749f8b1a28505148f07b8` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:57.884848 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42184de-512e-4423-a2eb-8f5f633cf22e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f02796bb02749f8b1a28505148f07b8)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:35:57.886424 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42184de-512e-4423-a2eb-8f5f633cf22e admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:35:57.995610 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-d42184de-512e-4423-a2eb-8f5f633cf22e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f02796bb02749f8b1a28505148f07b8', 'request_id': 'req-d42184de-512e-4423-a2eb-8f5f633cf22e'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:35:58.001688 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-d42184de-512e-4423-a2eb-8f5f633cf22e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f02796bb02749f8b1a28505148f07b8', 'request_id': 'req-d42184de-512e-4423-a2eb-8f5f633cf22e'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:35:58.007650 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-d42184de-512e-4423-a2eb-8f5f633cf22e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f02796bb02749f8b1a28505148f07b8', 'request_id': 'req-d42184de-512e-4423-a2eb-8f5f633cf22e'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:35:58.075032 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-d42184de-512e-4423-a2eb-8f5f633cf22e admin admin] The token cache is being invalidate because project 7f02796bb02749f8b1a28505148f07b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:35:58.075329 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-d42184de-512e-4423-a2eb-8f5f633cf22e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:35:58.075955 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-d42184de-512e-4423-a2eb-8f5f633cf22e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:35:58.077553 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1211/2405] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:35:57 2022] DELETE /identity/v3/projects/7f02796bb02749f8b1a28505148f07b8 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:35:58.083029 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310275e6-d0b7-4363-b10e-9e9ba72e5046 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:58.083493 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310275e6-d0b7-4363-b10e-9e9ba72e5046 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:58.086390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310275e6-d0b7-4363-b10e-9e9ba72e5046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:58.104163 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-310275e6-d0b7-4363-b10e-9e9ba72e5046 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:35:58.105368 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-310275e6-d0b7-4363-b10e-9e9ba72e5046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:35:58.172334 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1195/2406] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:35:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:35:58.180050 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5332d-7587-4367-860a-1b9b2a8bd713 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:58.180393 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5332d-7587-4367-860a-1b9b2a8bd713 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:58.180675 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5332d-7587-4367-860a-1b9b2a8bd713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:58.200768 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-bbe5332d-7587-4367-860a-1b9b2a8bd713 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:35:58.201965 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-bbe5332d-7587-4367-860a-1b9b2a8bd713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:35:58.210479 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1212/2407] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:35:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:35:58.214737 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310275e6-d0b7-4363-b10e-9e9ba72e5046 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:58.224474 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310275e6-d0b7-4363-b10e-9e9ba72e5046 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:35:58.229017 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda28277-b04d-4624-9445-9f86592f0a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:35:58.229627 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda28277-b04d-4624-9445-9f86592f0a6b admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:58.229908 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda28277-b04d-4624-9445-9f86592f0a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:58.230184 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda28277-b04d-4624-9445-9f86592f0a6b admin admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:58.230872 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda28277-b04d-4624-9445-9f86592f0a6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:35:58.231180 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda28277-b04d-4624-9445-9f86592f0a6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:35:58.232801 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda28277-b04d-4624-9445-9f86592f0a6b admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:35:58.244589 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1196/2408] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:35:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:35:58.248695 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5332d-7587-4367-860a-1b9b2a8bd713 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:58.258315 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5332d-7587-4367-860a-1b9b2a8bd713 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:35:58.263286 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0eeb92-07ea-4dbb-bfcb-5de981972215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:35:58.263918 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0eeb92-07ea-4dbb-bfcb-5de981972215 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:58.264245 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0eeb92-07ea-4dbb-bfcb-5de981972215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:58.264527 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0eeb92-07ea-4dbb-bfcb-5de981972215 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:58.265217 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0eeb92-07ea-4dbb-bfcb-5de981972215 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:35:58.267110 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0eeb92-07ea-4dbb-bfcb-5de981972215 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:35:58.339639 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1213/2409] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:35:58 2022] POST /identity/v3/projects => generated 385 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:35:58.343718 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda28277-b04d-4624-9445-9f86592f0a6b admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:58.353677 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda28277-b04d-4624-9445-9f86592f0a6b admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:35:58.365388 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:35:58.409990 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592cf910-5657-4f0e-be5e-a29472e673f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:35:58.410607 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592cf910-5657-4f0e-be5e-a29472e673f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:58.410893 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592cf910-5657-4f0e-be5e-a29472e673f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:58.411168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592cf910-5657-4f0e-be5e-a29472e673f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:58.411827 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592cf910-5657-4f0e-be5e-a29472e673f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:35:58.413785 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592cf910-5657-4f0e-be5e-a29472e673f8 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:35:58.511494 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1197/2410] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:35:58 2022] POST /identity/v3/users => generated 293 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:35:58.516169 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0eeb92-07ea-4dbb-bfcb-5de981972215 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:58.527599 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0eeb92-07ea-4dbb-bfcb-5de981972215 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:35:58.532276 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d30cfe9-7470-4c12-b790-429efb9cd417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:35:58.532883 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d30cfe9-7470-4c12-b790-429efb9cd417 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:58.533222 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d30cfe9-7470-4c12-b790-429efb9cd417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:58.533501 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d30cfe9-7470-4c12-b790-429efb9cd417 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:58.534193 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d30cfe9-7470-4c12-b790-429efb9cd417 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:35:58.535789 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d30cfe9-7470-4c12-b790-429efb9cd417 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:35:58.546186 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1214/2411] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:35:58.550630 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592cf910-5657-4f0e-be5e-a29472e673f8 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:58.560723 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592cf910-5657-4f0e-be5e-a29472e673f8 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:35:58.566135 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e652c5-a903-4683-b89d-d5fe38f6d233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:35:58.566747 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e652c5-a903-4683-b89d-d5fe38f6d233 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:58.567031 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e652c5-a903-4683-b89d-d5fe38f6d233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:58.567310 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e652c5-a903-4683-b89d-d5fe38f6d233 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:58.568003 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e652c5-a903-4683-b89d-d5fe38f6d233 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:35:58.569623 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e652c5-a903-4683-b89d-d5fe38f6d233 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:35:58.580306 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1198/2412] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:35:58.584399 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d30cfe9-7470-4c12-b790-429efb9cd417 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:58.594410 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d30cfe9-7470-4c12-b790-429efb9cd417 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:35:58.599620 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cbf675-b677-4dd8-b4fc-a5a8fcad847b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:35:58.600306 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cbf675-b677-4dd8-b4fc-a5a8fcad847b admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:58.600591 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cbf675-b677-4dd8-b4fc-a5a8fcad847b admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:58.600870 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cbf675-b677-4dd8-b4fc-a5a8fcad847b admin admin] PATH_INFO: `/v3/projects/8193f0c201e64d5eb5c4e40e29566bb9/users/cbadd638c68748228a2a4784289d696c/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:58.615921 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cbf675-b677-4dd8-b4fc-a5a8fcad847b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8193f0c201e64d5eb5c4e40e29566bb9, user_id=cbadd638c68748228a2a4784289d696c, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:35:58.618709 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cbf675-b677-4dd8-b4fc-a5a8fcad847b admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:35:58.686456 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1215/2413] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:35:58 2022] PUT /identity/v3/projects/8193f0c201e64d5eb5c4e40e29566bb9/users/cbadd638c68748228a2a4784289d696c/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:35:58.698000 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd0c1b-2f15-447c-ab35-f1bc6e9d0ece None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:58.698288 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd0c1b-2f15-447c-ab35-f1bc6e9d0ece None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:58.698596 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd0c1b-2f15-447c-ab35-f1bc6e9d0ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:58.726231 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-c4bd0c1b-2f15-447c-ab35-f1bc6e9d0ece None None] MFA Rules not processed for user `cbadd638c68748228a2a4784289d696c`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:35:58.727487 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-c4bd0c1b-2f15-447c-ab35-f1bc6e9d0ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:35:58.797830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1199/2414] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:35:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:35:58.803115 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cbf675-b677-4dd8-b4fc-a5a8fcad847b admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:58.864508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:35:58.875168 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cbf675-b677-4dd8-b4fc-a5a8fcad847b admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:35:58.880577 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c38914-4cd7-4b2d-944c-53ac17b41bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:35:58.881202 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c38914-4cd7-4b2d-944c-53ac17b41bfe admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:58.881489 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c38914-4cd7-4b2d-944c-53ac17b41bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:58.881770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c38914-4cd7-4b2d-944c-53ac17b41bfe admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:58.882437 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c38914-4cd7-4b2d-944c-53ac17b41bfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:35:58.884465 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c38914-4cd7-4b2d-944c-53ac17b41bfe admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:35:58.979307 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1216/2415] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:35:58 2022] POST /identity/v3/projects => generated 385 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:35:58.987532 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd0c1b-2f15-447c-ab35-f1bc6e9d0ece None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:58.998348 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd0c1b-2f15-447c-ab35-f1bc6e9d0ece None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:35:59.054753 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39525978-8844-40c0-ad2f-bd1388a0a5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:35:59.055299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39525978-8844-40c0-ad2f-bd1388a0a5ca admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:59.055600 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39525978-8844-40c0-ad2f-bd1388a0a5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:59.055863 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39525978-8844-40c0-ad2f-bd1388a0a5ca admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:59.056572 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39525978-8844-40c0-ad2f-bd1388a0a5ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:35:59.058485 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39525978-8844-40c0-ad2f-bd1388a0a5ca admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:35:59.127739 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1200/2416] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:35:58 2022] POST /identity/v3/users => generated 293 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:35:59.136765 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c38914-4cd7-4b2d-944c-53ac17b41bfe admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:59.147869 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c38914-4cd7-4b2d-944c-53ac17b41bfe admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:35:59.152647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe952b86-3f84-4dc6-93a3-067d37d6193a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:35:59.153254 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe952b86-3f84-4dc6-93a3-067d37d6193a admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:59.153539 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe952b86-3f84-4dc6-93a3-067d37d6193a admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:59.154781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe952b86-3f84-4dc6-93a3-067d37d6193a admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:59.154781 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe952b86-3f84-4dc6-93a3-067d37d6193a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:35:59.156213 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe952b86-3f84-4dc6-93a3-067d37d6193a admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:35:59.166857 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1217/2417] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:35:59.173825 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39525978-8844-40c0-ad2f-bd1388a0a5ca admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:59.183931 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39525978-8844-40c0-ad2f-bd1388a0a5ca admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:35:59.188523 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed696bf-2a17-432c-9af6-5586f4705fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:35:59.189120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed696bf-2a17-432c-9af6-5586f4705fda admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:59.189893 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed696bf-2a17-432c-9af6-5586f4705fda admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:59.190202 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed696bf-2a17-432c-9af6-5586f4705fda admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:59.190891 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed696bf-2a17-432c-9af6-5586f4705fda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:35:59.192504 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed696bf-2a17-432c-9af6-5586f4705fda admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:35:59.202216 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1201/2418] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:35:59.206727 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe952b86-3f84-4dc6-93a3-067d37d6193a admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:59.216995 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe952b86-3f84-4dc6-93a3-067d37d6193a admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:35:59.221616 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d037e06b-9884-483e-a3f2-43b48264ea7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:35:59.222272 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d037e06b-9884-483e-a3f2-43b48264ea7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:59.222558 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d037e06b-9884-483e-a3f2-43b48264ea7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:59.222839 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d037e06b-9884-483e-a3f2-43b48264ea7a admin admin] PATH_INFO: `/v3/projects/71b1c49a70e54f719b1caa2b33e995c2/users/04313e7e2e8a47249891221564dc72c1/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:59.236977 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d037e06b-9884-483e-a3f2-43b48264ea7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=71b1c49a70e54f719b1caa2b33e995c2, user_id=04313e7e2e8a47249891221564dc72c1, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:35:59.239611 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d037e06b-9884-483e-a3f2-43b48264ea7a admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:35:59.349672 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1218/2419] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:35:59 2022] PUT /identity/v3/projects/71b1c49a70e54f719b1caa2b33e995c2/users/04313e7e2e8a47249891221564dc72c1/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:35:59.359956 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917535d-39b8-46e1-b67d-20a2e37fed2b None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:59.360339 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917535d-39b8-46e1-b67d-20a2e37fed2b None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:59.360624 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917535d-39b8-46e1-b67d-20a2e37fed2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:59.368311 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:35:59.389762 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-3917535d-39b8-46e1-b67d-20a2e37fed2b None None] MFA Rules not processed for user `04313e7e2e8a47249891221564dc72c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:35:59.390967 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-3917535d-39b8-46e1-b67d-20a2e37fed2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:35:59.454951 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1202/2420] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:35:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:35:59.459470 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d037e06b-9884-483e-a3f2-43b48264ea7a admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:59.520716 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d037e06b-9884-483e-a3f2-43b48264ea7a admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:35:59.525387 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f306cf6-a799-4cca-8986-ace9de3f1339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:35:59.526002 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f306cf6-a799-4cca-8986-ace9de3f1339 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:59.526288 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f306cf6-a799-4cca-8986-ace9de3f1339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:59.526566 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f306cf6-a799-4cca-8986-ace9de3f1339 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:59.527239 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f306cf6-a799-4cca-8986-ace9de3f1339 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:35:59.529186 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f306cf6-a799-4cca-8986-ace9de3f1339 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:35:59.596936 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1219/2421] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:35:59 2022] POST /identity/v3/projects => generated 385 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:35:59.601205 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3917535d-39b8-46e1-b67d-20a2e37fed2b None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:59.612531 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3917535d-39b8-46e1-b67d-20a2e37fed2b None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:35:59.668242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db508e-5b6a-4756-805d-8c7773a5baad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:35:59.668864 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db508e-5b6a-4756-805d-8c7773a5baad admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:59.669151 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db508e-5b6a-4756-805d-8c7773a5baad admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:59.669433 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db508e-5b6a-4756-805d-8c7773a5baad admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:59.670098 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30db508e-5b6a-4756-805d-8c7773a5baad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:35:59.672054 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30db508e-5b6a-4756-805d-8c7773a5baad admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:35:59.739156 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1203/2422] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:35:59 2022] POST /identity/v3/users => generated 299 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:35:59.743173 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f306cf6-a799-4cca-8986-ace9de3f1339 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:59.753524 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f306cf6-a799-4cca-8986-ace9de3f1339 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:35:59.758197 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec743c0-c2ad-448a-a254-ce16de76d2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:35:59.758804 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec743c0-c2ad-448a-a254-ce16de76d2d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:59.759090 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec743c0-c2ad-448a-a254-ce16de76d2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:59.759370 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec743c0-c2ad-448a-a254-ce16de76d2d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:59.760100 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec743c0-c2ad-448a-a254-ce16de76d2d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:35:59.761685 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec743c0-c2ad-448a-a254-ce16de76d2d0 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:35:59.772418 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1220/2423] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:35:59.776329 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db508e-5b6a-4756-805d-8c7773a5baad admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:59.786884 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db508e-5b6a-4756-805d-8c7773a5baad admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:35:59.791481 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e95cc-5fd4-45d7-9659-92a8350e974f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:35:59.792162 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e95cc-5fd4-45d7-9659-92a8350e974f admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:59.792444 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e95cc-5fd4-45d7-9659-92a8350e974f admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:59.792716 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e95cc-5fd4-45d7-9659-92a8350e974f admin admin] PATH_INFO: `/v3/projects/f154c210d6374195829ac3dd8098a8af/users/46f76200e2724b619c4e296b186f9b4f/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:59.808690 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e95cc-5fd4-45d7-9659-92a8350e974f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f154c210d6374195829ac3dd8098a8af, user_id=46f76200e2724b619c4e296b186f9b4f, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:35:59.811345 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e95cc-5fd4-45d7-9659-92a8350e974f admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:35:59.898791 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1204/2424] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:35:59 2022] PUT /identity/v3/projects/f154c210d6374195829ac3dd8098a8af/users/46f76200e2724b619c4e296b186f9b4f/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:35:59.902811 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec743c0-c2ad-448a-a254-ce16de76d2d0 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:59.913434 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec743c0-c2ad-448a-a254-ce16de76d2d0 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:35:59.968952 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36976e11-075b-4384-84cd-5028813ab109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:35:59.969566 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36976e11-075b-4384-84cd-5028813ab109 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:35:59.969854 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36976e11-075b-4384-84cd-5028813ab109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:35:59.970137 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36976e11-075b-4384-84cd-5028813ab109 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:35:59.971325 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36976e11-075b-4384-84cd-5028813ab109 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:35:59.972978 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36976e11-075b-4384-84cd-5028813ab109 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:35:59.982638 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1221/2425] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:35:59.987400 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e95cc-5fd4-45d7-9659-92a8350e974f admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:35:59.997682 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e95cc-5fd4-45d7-9659-92a8350e974f admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:36:00.002330 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec2061-63cb-4c84-aba2-a869f76dac5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:36:00.002979 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec2061-63cb-4c84-aba2-a869f76dac5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:00.003261 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec2061-63cb-4c84-aba2-a869f76dac5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:00.003535 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec2061-63cb-4c84-aba2-a869f76dac5b admin admin] PATH_INFO: `/v3/projects/f154c210d6374195829ac3dd8098a8af/users/46f76200e2724b619c4e296b186f9b4f/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:00.005498 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ec2061-63cb-4c84-aba2-a869f76dac5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f154c210d6374195829ac3dd8098a8af, user_id=46f76200e2724b619c4e296b186f9b4f, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:36:00.008131 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ec2061-63cb-4c84-aba2-a869f76dac5b admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:36:00.018389 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1205/2426] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:35:59 2022] PUT /identity/v3/projects/f154c210d6374195829ac3dd8098a8af/users/46f76200e2724b619c4e296b186f9b4f/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:36:00.028620 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e75d7-1fe2-4d29-9f0b-d9f161127a6b None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:00.028916 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e75d7-1fe2-4d29-9f0b-d9f161127a6b None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:00.029205 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e75d7-1fe2-4d29-9f0b-d9f161127a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:00.055400 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:36:00.058299 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-896e75d7-1fe2-4d29-9f0b-d9f161127a6b None None] MFA Rules not processed for user `46f76200e2724b619c4e296b186f9b4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:36:00.059543 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-896e75d7-1fe2-4d29-9f0b-d9f161127a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:36:00.131323 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1222/2427] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:36:00 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:36:00.143574 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec2061-63cb-4c84-aba2-a869f76dac5b admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:36:00.145370 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-d2ec2061-63cb-4c84-aba2-a869f76dac5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:36:00.192977 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:36:00.203454 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec2061-63cb-4c84-aba2-a869f76dac5b admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:36:00.207703 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed578e39-fb98-4d80-8ebf-fdb7b0c6bcae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:36:00.208352 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed578e39-fb98-4d80-8ebf-fdb7b0c6bcae service cinder] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:00.208640 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed578e39-fb98-4d80-8ebf-fdb7b0c6bcae service cinder] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:00.208916 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed578e39-fb98-4d80-8ebf-fdb7b0c6bcae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:00.264535 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed578e39-fb98-4d80-8ebf-fdb7b0c6bcae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:36:00.265974 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed578e39-fb98-4d80-8ebf-fdb7b0c6bcae service cinder] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:36:00.275333 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1206/2428] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:36:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:36:02.231463 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e75d7-1fe2-4d29-9f0b-d9f161127a6b None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:36:02.241941 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e75d7-1fe2-4d29-9f0b-d9f161127a6b None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:36:02.245669 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cd5b06-88b2-40c6-b7e5-cc500c024e78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:36:02.246359 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd5b06-88b2-40c6-b7e5-cc500c024e78 service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:02.246713 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd5b06-88b2-40c6-b7e5-cc500c024e78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:02.247061 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd5b06-88b2-40c6-b7e5-cc500c024e78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:02.323712 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cd5b06-88b2-40c6-b7e5-cc500c024e78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:36:02.325330 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cd5b06-88b2-40c6-b7e5-cc500c024e78 service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:36:02.337358 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1223/2429] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:36:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:36:03.717113 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed578e39-fb98-4d80-8ebf-fdb7b0c6bcae service cinder] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:36:03.726216 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed578e39-fb98-4d80-8ebf-fdb7b0c6bcae service cinder] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:36:03.729830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b5971-879f-4d36-882f-91237d0b322e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:36:03.730438 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b5971-879f-4d36-882f-91237d0b322e service cinder] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:03.730723 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b5971-879f-4d36-882f-91237d0b322e service cinder] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:03.731000 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b5971-879f-4d36-882f-91237d0b322e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:03.740433 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209b5971-879f-4d36-882f-91237d0b322e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:36:03.741846 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209b5971-879f-4d36-882f-91237d0b322e service cinder] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:36:03.751270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1207/2430] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:36:03 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:36:07.702059 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f9c9d-6806-4eaa-83f0-8b0d8df71d3a None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:07.702638 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f9c9d-6806-4eaa-83f0-8b0d8df71d3a None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:07.702638 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f9c9d-6806-4eaa-83f0-8b0d8df71d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:07.722637 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-853f9c9d-6806-4eaa-83f0-8b0d8df71d3a None None] MFA Rules not processed for user `1782b4e84c9f484297e17c248738105c`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:36:07.723840 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-853f9c9d-6806-4eaa-83f0-8b0d8df71d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:36:07.777791 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1224/2431] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:36:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:36:08.091837 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b5971-879f-4d36-882f-91237d0b322e service cinder] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:36:08.093307 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-209b5971-879f-4d36-882f-91237d0b322e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:36:08.149369 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b5971-879f-4d36-882f-91237d0b322e service cinder] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:36:08.152931 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db621b3-4259-4160-b588-651cc872248c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:36:08.153543 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db621b3-4259-4160-b588-651cc872248c service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:08.153828 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db621b3-4259-4160-b588-651cc872248c service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:08.154107 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db621b3-4259-4160-b588-651cc872248c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:08.155830 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-6db621b3-4259-4160-b588-651cc872248c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:36:08.165317 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db621b3-4259-4160-b588-651cc872248c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:36:08.166771 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db621b3-4259-4160-b588-651cc872248c service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:36:08.176146 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1208/2432] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:36:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:36:08.350520 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853f9c9d-6806-4eaa-83f0-8b0d8df71d3a None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:36:08.397192 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:36:08.415786 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853f9c9d-6806-4eaa-83f0-8b0d8df71d3a None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:36:08.420597 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d771a1-be98-4790-aada-77a20323528e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:36:08.421241 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d771a1-be98-4790-aada-77a20323528e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:08.421528 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d771a1-be98-4790-aada-77a20323528e admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:08.421836 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d771a1-be98-4790-aada-77a20323528e admin admin] PATH_INFO: `/v3/users/cbadd638c68748228a2a4784289d696c` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:08.423152 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d771a1-be98-4790-aada-77a20323528e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbadd638c68748228a2a4784289d696c)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:36:08.425648 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d771a1-be98-4790-aada-77a20323528e admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:36:08.628450 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-50d771a1-be98-4790-aada-77a20323528e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbadd638c68748228a2a4784289d696c'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:36:08.629121 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-50d771a1-be98-4790-aada-77a20323528e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbadd638c68748228a2a4784289d696c'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:36:08.641755 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-50d771a1-be98-4790-aada-77a20323528e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbadd638c68748228a2a4784289d696c'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:36:08.739081 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-50d771a1-be98-4790-aada-77a20323528e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbadd638c68748228a2a4784289d696c'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:36:08.749592 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-50d771a1-be98-4790-aada-77a20323528e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbadd638c68748228a2a4784289d696c'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:36:08.756536 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1225/2433] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:36:08 2022] DELETE /identity/v3/users/cbadd638c68748228a2a4784289d696c => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:36:08.761083 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0fd982-c8ad-4ed7-936e-b81b983588e0 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:08.763068 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0fd982-c8ad-4ed7-936e-b81b983588e0 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:08.763068 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0fd982-c8ad-4ed7-936e-b81b983588e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:08.781497 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-bc0fd982-c8ad-4ed7-936e-b81b983588e0 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:36:08.782741 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-bc0fd982-c8ad-4ed7-936e-b81b983588e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:36:08.845359 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1209/2434] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:36:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:36:08.855606 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d771a1-be98-4790-aada-77a20323528e admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:36:08.857047 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-50d771a1-be98-4790-aada-77a20323528e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:36:08.915381 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d771a1-be98-4790-aada-77a20323528e admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:36:08.919034 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06698652-23b7-49bf-9d9f-66874388b503 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:36:08.919650 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06698652-23b7-49bf-9d9f-66874388b503 service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:08.921562 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06698652-23b7-49bf-9d9f-66874388b503 service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:08.921562 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06698652-23b7-49bf-9d9f-66874388b503 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:08.929810 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06698652-23b7-49bf-9d9f-66874388b503 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:36:08.931349 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06698652-23b7-49bf-9d9f-66874388b503 service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:36:08.940899 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1226/2435] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:36:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:36:08.981477 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0fd982-c8ad-4ed7-936e-b81b983588e0 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:36:08.982926 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-bc0fd982-c8ad-4ed7-936e-b81b983588e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:36:09.017836 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:36:09.066271 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0fd982-c8ad-4ed7-936e-b81b983588e0 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:36:09.071331 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd68f47-00b9-4999-a00d-765ae3c5fdc0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '107a7c18a2c6440b86e1b7c3e39cc724', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:36:09.071986 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd68f47-00b9-4999-a00d-765ae3c5fdc0 service placement] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:09.072301 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd68f47-00b9-4999-a00d-765ae3c5fdc0 service placement] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:09.072582 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd68f47-00b9-4999-a00d-765ae3c5fdc0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:09.074419 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-2fd68f47-00b9-4999-a00d-765ae3c5fdc0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:36:09.086586 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd68f47-00b9-4999-a00d-765ae3c5fdc0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:36:09.088250 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd68f47-00b9-4999-a00d-765ae3c5fdc0 service placement] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:36:09.098710 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1210/2436] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:36:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:36:09.364015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06698652-23b7-49bf-9d9f-66874388b503 service neutron] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:36:09.365600 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-06698652-23b7-49bf-9d9f-66874388b503 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:36:09.380030 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06698652-23b7-49bf-9d9f-66874388b503 service neutron] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:36:09.384787 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1a10a8-ec5b-4adf-bfcc-2f855ae90484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:36:09.385571 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a10a8-ec5b-4adf-bfcc-2f855ae90484 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:09.385857 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a10a8-ec5b-4adf-bfcc-2f855ae90484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:09.386134 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a10a8-ec5b-4adf-bfcc-2f855ae90484 admin admin] PATH_INFO: `/v3/projects/8193f0c201e64d5eb5c4e40e29566bb9` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:09.387368 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1a10a8-ec5b-4adf-bfcc-2f855ae90484 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8193f0c201e64d5eb5c4e40e29566bb9)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:36:09.389492 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1a10a8-ec5b-4adf-bfcc-2f855ae90484 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:36:09.523477 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-7e1a10a8-ec5b-4adf-bfcc-2f855ae90484 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8193f0c201e64d5eb5c4e40e29566bb9', 'request_id': 'req-7e1a10a8-ec5b-4adf-bfcc-2f855ae90484'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:36:09.530821 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-7e1a10a8-ec5b-4adf-bfcc-2f855ae90484 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8193f0c201e64d5eb5c4e40e29566bb9', 'request_id': 'req-7e1a10a8-ec5b-4adf-bfcc-2f855ae90484'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:36:09.535632 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:36:09.541743 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-7e1a10a8-ec5b-4adf-bfcc-2f855ae90484 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8193f0c201e64d5eb5c4e40e29566bb9', 'request_id': 'req-7e1a10a8-ec5b-4adf-bfcc-2f855ae90484'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:36:09.630237 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-7e1a10a8-ec5b-4adf-bfcc-2f855ae90484 admin admin] The token cache is being invalidate because project 8193f0c201e64d5eb5c4e40e29566bb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:36:09.630541 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-7e1a10a8-ec5b-4adf-bfcc-2f855ae90484 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:36:09.631208 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-7e1a10a8-ec5b-4adf-bfcc-2f855ae90484 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:36:09.633169 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1227/2437] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:36:09 2022] DELETE /identity/v3/projects/8193f0c201e64d5eb5c4e40e29566bb9 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:36:09.642758 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd68f47-00b9-4999-a00d-765ae3c5fdc0 service placement] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:36:09.653527 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd68f47-00b9-4999-a00d-765ae3c5fdc0 service placement] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:36:09.685824 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1a10a8-ec5b-4adf-bfcc-2f855ae90484 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:36:09.687223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-7e1a10a8-ec5b-4adf-bfcc-2f855ae90484 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:36:09.712438 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663ac7d-ac0f-49b2-acc5-af8966d1fdec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:36:09.713142 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663ac7d-ac0f-49b2-acc5-af8966d1fdec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:09.713543 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663ac7d-ac0f-49b2-acc5-af8966d1fdec admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:09.713825 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663ac7d-ac0f-49b2-acc5-af8966d1fdec admin admin] PATH_INFO: `/v3/users/04313e7e2e8a47249891221564dc72c1` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:09.715045 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9663ac7d-ac0f-49b2-acc5-af8966d1fdec admin admin] RBAC: Authorizing `identity:delete_user(user_id=04313e7e2e8a47249891221564dc72c1)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:36:09.717085 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9663ac7d-ac0f-49b2-acc5-af8966d1fdec admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:36:09.762924 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1a10a8-ec5b-4adf-bfcc-2f855ae90484 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:36:09.769729 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48819c81-c9c6-4cec-bceb-61368b159b8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:36:09.776680 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48819c81-c9c6-4cec-bceb-61368b159b8d service nova] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:09.776680 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48819c81-c9c6-4cec-bceb-61368b159b8d service nova] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:09.776680 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48819c81-c9c6-4cec-bceb-61368b159b8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:09.776680 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-48819c81-c9c6-4cec-bceb-61368b159b8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:36:09.830180 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48819c81-c9c6-4cec-bceb-61368b159b8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:36:09.832489 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48819c81-c9c6-4cec-bceb-61368b159b8d service nova] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:36:09.841817 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1228/2438] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:36:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:36:09.935493 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-9663ac7d-ac0f-49b2-acc5-af8966d1fdec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04313e7e2e8a47249891221564dc72c1'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:36:09.944194 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-9663ac7d-ac0f-49b2-acc5-af8966d1fdec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04313e7e2e8a47249891221564dc72c1'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:36:09.948880 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-9663ac7d-ac0f-49b2-acc5-af8966d1fdec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04313e7e2e8a47249891221564dc72c1'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:36:09.957932 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-9663ac7d-ac0f-49b2-acc5-af8966d1fdec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04313e7e2e8a47249891221564dc72c1'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:36:09.958598 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-9663ac7d-ac0f-49b2-acc5-af8966d1fdec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04313e7e2e8a47249891221564dc72c1'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:36:10.025129 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1211/2439] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:36:09 2022] DELETE /identity/v3/users/04313e7e2e8a47249891221564dc72c1 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:36:10.443977 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48819c81-c9c6-4cec-bceb-61368b159b8d service nova] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:36:10.445659 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-48819c81-c9c6-4cec-bceb-61368b159b8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:36:10.469731 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48819c81-c9c6-4cec-bceb-61368b159b8d service nova] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:36:10.540114 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf54219d-f5ef-49bf-83c6-ae0ffdc58b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:36:10.540769 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54219d-f5ef-49bf-83c6-ae0ffdc58b1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:10.541055 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54219d-f5ef-49bf-83c6-ae0ffdc58b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:10.541429 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54219d-f5ef-49bf-83c6-ae0ffdc58b1e admin admin] PATH_INFO: `/v3/projects/71b1c49a70e54f719b1caa2b33e995c2` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:10.542838 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf54219d-f5ef-49bf-83c6-ae0ffdc58b1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=71b1c49a70e54f719b1caa2b33e995c2)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:36:10.545938 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf54219d-f5ef-49bf-83c6-ae0ffdc58b1e admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:36:10.644368 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:36:10.654594 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663ac7d-ac0f-49b2-acc5-af8966d1fdec admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:36:10.656239 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-9663ac7d-ac0f-49b2-acc5-af8966d1fdec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:36:10.661947 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-bf54219d-f5ef-49bf-83c6-ae0ffdc58b1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71b1c49a70e54f719b1caa2b33e995c2', 'request_id': 'req-bf54219d-f5ef-49bf-83c6-ae0ffdc58b1e'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:36:10.669031 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-bf54219d-f5ef-49bf-83c6-ae0ffdc58b1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71b1c49a70e54f719b1caa2b33e995c2', 'request_id': 'req-bf54219d-f5ef-49bf-83c6-ae0ffdc58b1e'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:36:10.688250 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-bf54219d-f5ef-49bf-83c6-ae0ffdc58b1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71b1c49a70e54f719b1caa2b33e995c2', 'request_id': 'req-bf54219d-f5ef-49bf-83c6-ae0ffdc58b1e'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:36:10.698996 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:36:10.732106 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663ac7d-ac0f-49b2-acc5-af8966d1fdec admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:36:10.746842 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-bf54219d-f5ef-49bf-83c6-ae0ffdc58b1e admin admin] The token cache is being invalidate because project 71b1c49a70e54f719b1caa2b33e995c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:36:10.747137 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-bf54219d-f5ef-49bf-83c6-ae0ffdc58b1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:36:10.747933 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-bf54219d-f5ef-49bf-83c6-ae0ffdc58b1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:36:10.749849 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1229/2440] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:36:10 2022] DELETE /identity/v3/projects/71b1c49a70e54f719b1caa2b33e995c2 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:36:10.753659 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf54219d-f5ef-49bf-83c6-ae0ffdc58b1e admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:36:10.755120 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-bf54219d-f5ef-49bf-83c6-ae0ffdc58b1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:36:10.798477 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6908d4d6-a0e1-49d8-bcfb-008118354d7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:36:10.799098 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908d4d6-a0e1-49d8-bcfb-008118354d7d service nova] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:10.799410 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908d4d6-a0e1-49d8-bcfb-008118354d7d service nova] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:10.799690 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908d4d6-a0e1-49d8-bcfb-008118354d7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:10.801550 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-6908d4d6-a0e1-49d8-bcfb-008118354d7d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:36:10.820731 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf54219d-f5ef-49bf-83c6-ae0ffdc58b1e admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:36:10.825573 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b740fdc-7a7c-4100-bf5a-159608d93515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:36:10.826245 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b740fdc-7a7c-4100-bf5a-159608d93515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:10.826514 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b740fdc-7a7c-4100-bf5a-159608d93515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:10.826833 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b740fdc-7a7c-4100-bf5a-159608d93515 admin admin] PATH_INFO: `/v3/users/46f76200e2724b619c4e296b186f9b4f` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:10.828001 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b740fdc-7a7c-4100-bf5a-159608d93515 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46f76200e2724b619c4e296b186f9b4f)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:36:10.830020 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b740fdc-7a7c-4100-bf5a-159608d93515 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:36:10.876535 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6908d4d6-a0e1-49d8-bcfb-008118354d7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:36:10.878775 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6908d4d6-a0e1-49d8-bcfb-008118354d7d service nova] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:36:10.888331 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1212/2441] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:36:10 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 15:36:11.074111 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-5b740fdc-7a7c-4100-bf5a-159608d93515 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46f76200e2724b619c4e296b186f9b4f'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:36:11.074860 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-5b740fdc-7a7c-4100-bf5a-159608d93515 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46f76200e2724b619c4e296b186f9b4f'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:36:11.088675 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-5b740fdc-7a7c-4100-bf5a-159608d93515 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46f76200e2724b619c4e296b186f9b4f'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:36:11.175223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-5b740fdc-7a7c-4100-bf5a-159608d93515 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46f76200e2724b619c4e296b186f9b4f'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:36:11.185120 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-5b740fdc-7a7c-4100-bf5a-159608d93515 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46f76200e2724b619c4e296b186f9b4f'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:36:11.195710 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1230/2442] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:36:10 2022] DELETE /identity/v3/users/46f76200e2724b619c4e296b186f9b4f => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:36:11.708316 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6908d4d6-a0e1-49d8-bcfb-008118354d7d service nova] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:36:11.720828 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6908d4d6-a0e1-49d8-bcfb-008118354d7d service nova] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:36:11.807779 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27446272-b8a0-46da-9230-137ad9a62318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:36:11.808640 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27446272-b8a0-46da-9230-137ad9a62318 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:11.808932 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27446272-b8a0-46da-9230-137ad9a62318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:11.809212 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27446272-b8a0-46da-9230-137ad9a62318 admin admin] PATH_INFO: `/v3/projects/f154c210d6374195829ac3dd8098a8af` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:11.810395 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27446272-b8a0-46da-9230-137ad9a62318 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f154c210d6374195829ac3dd8098a8af)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:36:11.812735 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27446272-b8a0-46da-9230-137ad9a62318 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:36:11.831339 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:36:11.935924 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-27446272-b8a0-46da-9230-137ad9a62318 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f154c210d6374195829ac3dd8098a8af', 'request_id': 'req-27446272-b8a0-46da-9230-137ad9a62318'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:36:11.942322 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-27446272-b8a0-46da-9230-137ad9a62318 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f154c210d6374195829ac3dd8098a8af', 'request_id': 'req-27446272-b8a0-46da-9230-137ad9a62318'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:36:12.073790 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-27446272-b8a0-46da-9230-137ad9a62318 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f154c210d6374195829ac3dd8098a8af', 'request_id': 'req-27446272-b8a0-46da-9230-137ad9a62318'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:36:12.091139 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-27446272-b8a0-46da-9230-137ad9a62318 admin admin] The token cache is being invalidate because project f154c210d6374195829ac3dd8098a8af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:36:12.091565 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-27446272-b8a0-46da-9230-137ad9a62318 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:36:12.092337 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-27446272-b8a0-46da-9230-137ad9a62318 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:36:12.094054 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1213/2443] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:36:11 2022] DELETE /identity/v3/projects/f154c210d6374195829ac3dd8098a8af => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:36:12.099828 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52afa933-38e2-4815-8652-ab4f850359b8 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:12.100388 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52afa933-38e2-4815-8652-ab4f850359b8 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:12.100534 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52afa933-38e2-4815-8652-ab4f850359b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:12.121614 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-52afa933-38e2-4815-8652-ab4f850359b8 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:36:12.122994 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-52afa933-38e2-4815-8652-ab4f850359b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:36:12.163857 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:36:12.206157 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1231/2444] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:36:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:36:12.213904 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94944f5-4911-4857-85e5-bf51a2e198fe None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:12.214269 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94944f5-4911-4857-85e5-bf51a2e198fe None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:12.214617 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94944f5-4911-4857-85e5-bf51a2e198fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:12.234774 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-f94944f5-4911-4857-85e5-bf51a2e198fe None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:36:12.236144 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-f94944f5-4911-4857-85e5-bf51a2e198fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:36:12.245197 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1214/2445] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:36:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:36:12.249755 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52afa933-38e2-4815-8652-ab4f850359b8 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:36:12.267671 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52afa933-38e2-4815-8652-ab4f850359b8 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:36:12.276550 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ffca8e-3187-4bcb-8ac4-51ab5785c5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:36:12.280427 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffca8e-3187-4bcb-8ac4-51ab5785c5f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:12.280427 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffca8e-3187-4bcb-8ac4-51ab5785c5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:12.280427 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffca8e-3187-4bcb-8ac4-51ab5785c5f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:12.280427 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ffca8e-3187-4bcb-8ac4-51ab5785c5f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:36:12.284368 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ffca8e-3187-4bcb-8ac4-51ab5785c5f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:36:12.286419 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ffca8e-3187-4bcb-8ac4-51ab5785c5f7 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:36:12.301806 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1232/2446] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:36:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:36:12.306793 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94944f5-4911-4857-85e5-bf51a2e198fe None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:36:12.318671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94944f5-4911-4857-85e5-bf51a2e198fe None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:36:12.324330 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e30a3eb-91b4-4ea8-8d5b-f35dea92e7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:36:12.324959 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e30a3eb-91b4-4ea8-8d5b-f35dea92e7a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:12.325385 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e30a3eb-91b4-4ea8-8d5b-f35dea92e7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:12.325667 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e30a3eb-91b4-4ea8-8d5b-f35dea92e7a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:12.326772 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e30a3eb-91b4-4ea8-8d5b-f35dea92e7a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:36:12.328839 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e30a3eb-91b4-4ea8-8d5b-f35dea92e7a4 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:36:12.446683 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1215/2447] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:36:12 2022] POST /identity/v3/projects => generated 373 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:36:12.452840 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ffca8e-3187-4bcb-8ac4-51ab5785c5f7 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:36:12.468046 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ffca8e-3187-4bcb-8ac4-51ab5785c5f7 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:36:12.554574 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ccc6de-f01f-45bf-b3f4-0f534cdc698e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:36:12.555210 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ccc6de-f01f-45bf-b3f4-0f534cdc698e admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:12.555495 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ccc6de-f01f-45bf-b3f4-0f534cdc698e admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:12.555775 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ccc6de-f01f-45bf-b3f4-0f534cdc698e admin admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:12.557084 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ccc6de-f01f-45bf-b3f4-0f534cdc698e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:36:12.559043 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ccc6de-f01f-45bf-b3f4-0f534cdc698e admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:36:12.666727 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1233/2448] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:36:12 2022] POST /identity/v3/users => generated 287 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:36:12.668171 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e30a3eb-91b4-4ea8-8d5b-f35dea92e7a4 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:36:12.679311 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e30a3eb-91b4-4ea8-8d5b-f35dea92e7a4 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:36:12.684368 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4610af-18c8-499d-9240-ff43bbcc7413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:36:12.685301 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4610af-18c8-499d-9240-ff43bbcc7413 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:12.685531 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4610af-18c8-499d-9240-ff43bbcc7413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:12.685626 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4610af-18c8-499d-9240-ff43bbcc7413 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:12.686297 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4610af-18c8-499d-9240-ff43bbcc7413 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:36:12.687935 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4610af-18c8-499d-9240-ff43bbcc7413 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:36:12.698360 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1216/2449] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:36:12.702630 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ccc6de-f01f-45bf-b3f4-0f534cdc698e admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:36:12.713960 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ccc6de-f01f-45bf-b3f4-0f534cdc698e admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:36:12.717807 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a3de1-bc32-49ca-a7d0-c9b6658538d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:36:12.718461 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a3de1-bc32-49ca-a7d0-c9b6658538d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:12.718729 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a3de1-bc32-49ca-a7d0-c9b6658538d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:12.719009 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a3de1-bc32-49ca-a7d0-c9b6658538d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:12.719727 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072a3de1-bc32-49ca-a7d0-c9b6658538d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:36:12.721896 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072a3de1-bc32-49ca-a7d0-c9b6658538d9 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:36:12.733550 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1234/2450] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 15:36:12.738297 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4610af-18c8-499d-9240-ff43bbcc7413 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:36:12.748733 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4610af-18c8-499d-9240-ff43bbcc7413 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:36:12.753556 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5adb8c-0d0d-41e4-b330-ad6d3250ad69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:36:12.754214 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5adb8c-0d0d-41e4-b330-ad6d3250ad69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:12.754562 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5adb8c-0d0d-41e4-b330-ad6d3250ad69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:12.754790 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5adb8c-0d0d-41e4-b330-ad6d3250ad69 admin admin] PATH_INFO: `/v3/projects/07cd370ebabf4f048e72f7b1fbadcb08/users/ff040e1f99bf46d48446e2e9c0deb9cc/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:12.769655 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5adb8c-0d0d-41e4-b330-ad6d3250ad69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07cd370ebabf4f048e72f7b1fbadcb08, user_id=ff040e1f99bf46d48446e2e9c0deb9cc, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:36:12.772302 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5adb8c-0d0d-41e4-b330-ad6d3250ad69 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:36:12.878055 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1217/2451] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:36:12 2022] PUT /identity/v3/projects/07cd370ebabf4f048e72f7b1fbadcb08/users/ff040e1f99bf46d48446e2e9c0deb9cc/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:36:12.883192 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740dac60-aa61-488f-be27-e87ecf94cc8c None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:12.883487 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740dac60-aa61-488f-be27-e87ecf94cc8c None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:12.883771 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740dac60-aa61-488f-be27-e87ecf94cc8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:12.903366 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-740dac60-aa61-488f-be27-e87ecf94cc8c None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:36:12.904636 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-740dac60-aa61-488f-be27-e87ecf94cc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:36:12.973714 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1235/2452] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:36:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:36:12.984368 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5adb8c-0d0d-41e4-b330-ad6d3250ad69 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:36:12.986065 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-eb5adb8c-0d0d-41e4-b330-ad6d3250ad69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:36:13.046150 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5adb8c-0d0d-41e4-b330-ad6d3250ad69 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:36:13.050386 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c9309-a291-47fb-8702-119769449ab4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:36:13.051259 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c9309-a291-47fb-8702-119769449ab4 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:13.051545 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c9309-a291-47fb-8702-119769449ab4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:13.051821 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c9309-a291-47fb-8702-119769449ab4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:13.064450 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c9309-a291-47fb-8702-119769449ab4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:36:13.064450 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c9309-a291-47fb-8702-119769449ab4 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:36:13.074894 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1218/2453] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:36:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:36:13.400757 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740dac60-aa61-488f-be27-e87ecf94cc8c None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:36:13.411003 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740dac60-aa61-488f-be27-e87ecf94cc8c None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:36:13.420141 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c6371-9697-4ec8-8aea-514546f7c364 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:36:13.420141 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c6371-9697-4ec8-8aea-514546f7c364 service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:13.420141 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c6371-9697-4ec8-8aea-514546f7c364 service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:13.420141 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c6371-9697-4ec8-8aea-514546f7c364 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:13.421626 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-1f4c6371-9697-4ec8-8aea-514546f7c364 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:36:13.433332 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4c6371-9697-4ec8-8aea-514546f7c364 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:36:13.435082 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4c6371-9697-4ec8-8aea-514546f7c364 service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:36:13.445236 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1236/2454] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:36:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:36:15.034590 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b5dcc-9d88-451a-9d79-a29da1eed0f2 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:15.035221 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b5dcc-9d88-451a-9d79-a29da1eed0f2 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:15.035221 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b5dcc-9d88-451a-9d79-a29da1eed0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:15.064673 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-f65b5dcc-9d88-451a-9d79-a29da1eed0f2 None None] MFA Rules not processed for user `ff040e1f99bf46d48446e2e9c0deb9cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:36:15.066399 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-f65b5dcc-9d88-451a-9d79-a29da1eed0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:36:15.097013 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:36:15.132363 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1219/2455] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:36:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:36:15.143807 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c6371-9697-4ec8-8aea-514546f7c364 service neutron] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:36:15.146629 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-1f4c6371-9697-4ec8-8aea-514546f7c364 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:36:15.177187 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:36:15.204639 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c6371-9697-4ec8-8aea-514546f7c364 service neutron] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:36:15.208950 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c2391-e3f6-4529-946c-c3c727776a8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:36:15.214653 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c2391-e3f6-4529-946c-c3c727776a8c service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:15.214653 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c2391-e3f6-4529-946c-c3c727776a8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:15.214653 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c2391-e3f6-4529-946c-c3c727776a8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:15.220425 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50c2391-e3f6-4529-946c-c3c727776a8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:36:15.221958 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50c2391-e3f6-4529-946c-c3c727776a8c service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:36:15.233091 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1237/2456] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:36:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:36:16.920856 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e30cd5-ceb5-4813-a376-66bd401c8f87 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:16.921358 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e30cd5-ceb5-4813-a376-66bd401c8f87 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:16.921465 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e30cd5-ceb5-4813-a376-66bd401c8f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:16.939891 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-f7e30cd5-ceb5-4813-a376-66bd401c8f87 None None] MFA Rules not processed for user `ff040e1f99bf46d48446e2e9c0deb9cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:36:16.941661 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-f7e30cd5-ceb5-4813-a376-66bd401c8f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:36:16.953955 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1220/2457] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:36:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:36:22.334452 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c2391-e3f6-4529-946c-c3c727776a8c service cinder] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:36:22.336646 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-c50c2391-e3f6-4529-946c-c3c727776a8c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:36:22.434402 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c2391-e3f6-4529-946c-c3c727776a8c service cinder] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:36:22.441450 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5021690-7ce6-4f13-92d6-2a1455f43c79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:36:22.442378 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5021690-7ce6-4f13-92d6-2a1455f43c79 service nova] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:22.442518 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5021690-7ce6-4f13-92d6-2a1455f43c79 service nova] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:22.442769 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5021690-7ce6-4f13-92d6-2a1455f43c79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:22.444743 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-d5021690-7ce6-4f13-92d6-2a1455f43c79 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:36:22.468209 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5021690-7ce6-4f13-92d6-2a1455f43c79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:36:22.468209 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5021690-7ce6-4f13-92d6-2a1455f43c79 service nova] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:36:22.472146 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1238/2458] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:36:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:36:22.866444 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80101e50-ba95-4a11-a7ce-decb97ca5c3d None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:22.866940 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80101e50-ba95-4a11-a7ce-decb97ca5c3d None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:22.867073 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80101e50-ba95-4a11-a7ce-decb97ca5c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:22.888223 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-80101e50-ba95-4a11-a7ce-decb97ca5c3d None None] MFA Rules not processed for user `1654fb6a81e14d6b99ab495bb6d029d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:36:22.889448 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-80101e50-ba95-4a11-a7ce-decb97ca5c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:36:22.969696 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1221/2459] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 5 15:36:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:36:23.105543 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5021690-7ce6-4f13-92d6-2a1455f43c79 service nova] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:36:23.106223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-d5021690-7ce6-4f13-92d6-2a1455f43c79 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:36:23.178740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5021690-7ce6-4f13-92d6-2a1455f43c79 service nova] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:36:23.188474 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31700852-24c2-45d5-a1c7-ab72106a2301 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85154ce7446245738edcc8702dbf6861', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:36:23.189116 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31700852-24c2-45d5-a1c7-ab72106a2301 service swift] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:23.189400 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31700852-24c2-45d5-a1c7-ab72106a2301 service swift] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:23.189730 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31700852-24c2-45d5-a1c7-ab72106a2301 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:23.198928 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31700852-24c2-45d5-a1c7-ab72106a2301 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:36:23.200734 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31700852-24c2-45d5-a1c7-ab72106a2301 service swift] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:36:23.209288 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1239/2460] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 5 15:36:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 05 15:36:32.961650 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=41 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:36:33.206287 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=80 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:36:40.201060 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80101e50-ba95-4a11-a7ce-decb97ca5c3d None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:36:40.226125 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80101e50-ba95-4a11-a7ce-decb97ca5c3d None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:36:40.234939 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afa3c9d-11ac-46fc-9c19-2005579cf13e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:36:40.235798 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa3c9d-11ac-46fc-9c19-2005579cf13e service nova] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:40.236896 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa3c9d-11ac-46fc-9c19-2005579cf13e service nova] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:40.237252 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa3c9d-11ac-46fc-9c19-2005579cf13e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:40.239130 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-3afa3c9d-11ac-46fc-9c19-2005579cf13e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:36:40.253097 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afa3c9d-11ac-46fc-9c19-2005579cf13e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:36:40.254763 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afa3c9d-11ac-46fc-9c19-2005579cf13e service nova] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:36:40.267248 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1222/2461] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:36:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 15:36:41.237629 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e3ee6-54ca-4e37-9ffb-b173a840b919 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:41.238295 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e3ee6-54ca-4e37-9ffb-b173a840b919 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:41.238295 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e3ee6-54ca-4e37-9ffb-b173a840b919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:41.258560 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-0a8e3ee6-54ca-4e37-9ffb-b173a840b919 None None] MFA Rules not processed for user `ff040e1f99bf46d48446e2e9c0deb9cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:36:41.259790 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-0a8e3ee6-54ca-4e37-9ffb-b173a840b919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:36:41.268391 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1240/2462] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:36:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:36:41.635562 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afa3c9d-11ac-46fc-9c19-2005579cf13e service nova] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:36:41.647340 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afa3c9d-11ac-46fc-9c19-2005579cf13e service nova] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:36:41.652393 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d74bf-92aa-4813-b1d1-e55bcfe3f708 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:36:41.653259 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d74bf-92aa-4813-b1d1-e55bcfe3f708 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:41.653545 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d74bf-92aa-4813-b1d1-e55bcfe3f708 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:41.653822 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d74bf-92aa-4813-b1d1-e55bcfe3f708 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:41.655567 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-d19d74bf-92aa-4813-b1d1-e55bcfe3f708 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:36:41.671544 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d74bf-92aa-4813-b1d1-e55bcfe3f708 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:36:41.671544 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d74bf-92aa-4813-b1d1-e55bcfe3f708 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:36:41.678396 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1223/2463] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:36:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:36:41.755468 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e3ee6-54ca-4e37-9ffb-b173a840b919 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:36:41.774011 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e3ee6-54ca-4e37-9ffb-b173a840b919 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:36:41.788877 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246affe8-6e24-4e82-bc32-e2a401b378f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:36:41.789689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246affe8-6e24-4e82-bc32-e2a401b378f0 service nova] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:41.790444 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246affe8-6e24-4e82-bc32-e2a401b378f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:41.791517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246affe8-6e24-4e82-bc32-e2a401b378f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:41.794159 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-246affe8-6e24-4e82-bc32-e2a401b378f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:36:41.813293 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246affe8-6e24-4e82-bc32-e2a401b378f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:36:41.815117 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246affe8-6e24-4e82-bc32-e2a401b378f0 service nova] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:36:41.848663 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1241/2464] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:36:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:36:45.617382 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab7f9d-8836-403f-9e25-54d362128843 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:45.617914 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab7f9d-8836-403f-9e25-54d362128843 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:45.618404 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab7f9d-8836-403f-9e25-54d362128843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:45.640678 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-1aab7f9d-8836-403f-9e25-54d362128843 None None] MFA Rules not processed for user `1654fb6a81e14d6b99ab495bb6d029d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:36:45.642851 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-1aab7f9d-8836-403f-9e25-54d362128843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:36:45.650948 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1224/2465] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 5 15:36:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:36:45.673089 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246affe8-6e24-4e82-bc32-e2a401b378f0 service nova] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:36:45.686923 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246affe8-6e24-4e82-bc32-e2a401b378f0 service nova] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:36:45.692907 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68690c99-2c4e-4d88-acd2-3cc68d122fb8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85154ce7446245738edcc8702dbf6861', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:36:45.693543 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68690c99-2c4e-4d88-acd2-3cc68d122fb8 service swift] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:45.693831 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68690c99-2c4e-4d88-acd2-3cc68d122fb8 service swift] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:45.694111 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68690c99-2c4e-4d88-acd2-3cc68d122fb8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:45.704797 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68690c99-2c4e-4d88-acd2-3cc68d122fb8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:36:45.706992 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68690c99-2c4e-4d88-acd2-3cc68d122fb8 service swift] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:36:45.718054 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1242/2466] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 5 15:36:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 15:36:47.554556 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab7f9d-8836-403f-9e25-54d362128843 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:36:47.566343 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab7f9d-8836-403f-9e25-54d362128843 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:36:47.570161 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98431f-3263-4f76-923b-0710dc40a67f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:36:47.570805 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98431f-3263-4f76-923b-0710dc40a67f service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:47.571094 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98431f-3263-4f76-923b-0710dc40a67f service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:47.571374 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98431f-3263-4f76-923b-0710dc40a67f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:47.573487 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-8e98431f-3263-4f76-923b-0710dc40a67f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:36:47.587317 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e98431f-3263-4f76-923b-0710dc40a67f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:36:47.588814 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e98431f-3263-4f76-923b-0710dc40a67f service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:36:47.599583 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1225/2467] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:36:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 15:36:52.412415 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68690c99-2c4e-4d88-acd2-3cc68d122fb8 service swift] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:36:52.414498 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-68690c99-2c4e-4d88-acd2-3cc68d122fb8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:36:52.521378 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98431f-3263-4f76-923b-0710dc40a67f service neutron] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:36:52.527287 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-8e98431f-3263-4f76-923b-0710dc40a67f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:36:52.527527 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68690c99-2c4e-4d88-acd2-3cc68d122fb8 service swift] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:36:52.529282 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af068b54-0dc1-487a-b4c1-4eaf5eb7944d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '107a7c18a2c6440b86e1b7c3e39cc724', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:36:52.529920 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af068b54-0dc1-487a-b4c1-4eaf5eb7944d service placement] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:52.530208 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af068b54-0dc1-487a-b4c1-4eaf5eb7944d service placement] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:52.530508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af068b54-0dc1-487a-b4c1-4eaf5eb7944d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:52.532353 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-af068b54-0dc1-487a-b4c1-4eaf5eb7944d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:36:52.544031 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98431f-3263-4f76-923b-0710dc40a67f service neutron] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:36:52.547895 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af068b54-0dc1-487a-b4c1-4eaf5eb7944d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:36:52.549730 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af068b54-0dc1-487a-b4c1-4eaf5eb7944d service placement] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:36:52.550798 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a72bf71-e633-4dd4-b3e2-75aef9377fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:36:52.554359 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72bf71-e633-4dd4-b3e2-75aef9377fc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:52.554866 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72bf71-e633-4dd4-b3e2-75aef9377fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:52.555730 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72bf71-e633-4dd4-b3e2-75aef9377fc8 admin admin] PATH_INFO: `/v3/users/04e80222936d4427909414a78e1dc489` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:52.556819 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a72bf71-e633-4dd4-b3e2-75aef9377fc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04e80222936d4427909414a78e1dc489)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:36:52.559078 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a72bf71-e633-4dd4-b3e2-75aef9377fc8 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:36:52.562468 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1243/2468] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:36:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:36:53.037339 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-8a72bf71-e633-4dd4-b3e2-75aef9377fc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04e80222936d4427909414a78e1dc489'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:36:53.045548 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-8a72bf71-e633-4dd4-b3e2-75aef9377fc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04e80222936d4427909414a78e1dc489'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:36:53.047433 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-8a72bf71-e633-4dd4-b3e2-75aef9377fc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04e80222936d4427909414a78e1dc489'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:36:53.061138 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-8a72bf71-e633-4dd4-b3e2-75aef9377fc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04e80222936d4427909414a78e1dc489'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:36:53.061957 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-8a72bf71-e633-4dd4-b3e2-75aef9377fc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04e80222936d4427909414a78e1dc489'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:36:53.131269 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1226/2469] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:36:52 2022] DELETE /identity/v3/users/04e80222936d4427909414a78e1dc489 => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:36:53.378224 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af068b54-0dc1-487a-b4c1-4eaf5eb7944d service placement] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:36:53.390943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af068b54-0dc1-487a-b4c1-4eaf5eb7944d service placement] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:36:53.483913 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ede166-8049-465e-a208-403331318dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:36:53.483913 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ede166-8049-465e-a208-403331318dd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:53.483913 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ede166-8049-465e-a208-403331318dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:53.483913 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ede166-8049-465e-a208-403331318dd2 admin admin] PATH_INFO: `/v3/projects/f9776de9e6d341fdbf7ac4fa70b994aa` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:53.494895 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ede166-8049-465e-a208-403331318dd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9776de9e6d341fdbf7ac4fa70b994aa)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:36:53.495780 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ede166-8049-465e-a208-403331318dd2 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:36:53.671229 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c3ede166-8049-465e-a208-403331318dd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9776de9e6d341fdbf7ac4fa70b994aa', 'request_id': 'req-c3ede166-8049-465e-a208-403331318dd2'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:36:53.680930 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c3ede166-8049-465e-a208-403331318dd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9776de9e6d341fdbf7ac4fa70b994aa', 'request_id': 'req-c3ede166-8049-465e-a208-403331318dd2'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:36:53.685532 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:36:53.704317 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c3ede166-8049-465e-a208-403331318dd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9776de9e6d341fdbf7ac4fa70b994aa', 'request_id': 'req-c3ede166-8049-465e-a208-403331318dd2'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:36:53.799253 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c3ede166-8049-465e-a208-403331318dd2 admin admin] The token cache is being invalidate because project f9776de9e6d341fdbf7ac4fa70b994aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:36:53.799633 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c3ede166-8049-465e-a208-403331318dd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:36:53.800258 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c3ede166-8049-465e-a208-403331318dd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:36:53.801920 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1244/2470] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:36:53 2022] DELETE /identity/v3/projects/f9776de9e6d341fdbf7ac4fa70b994aa => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:36:53.807651 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af270b2-8b68-4d3f-a7a0-1c7f2a850cfd None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:53.808240 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af270b2-8b68-4d3f-a7a0-1c7f2a850cfd None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:53.808348 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af270b2-8b68-4d3f-a7a0-1c7f2a850cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:53.830631 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-2af270b2-8b68-4d3f-a7a0-1c7f2a850cfd None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:36:53.831976 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-2af270b2-8b68-4d3f-a7a0-1c7f2a850cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:36:53.904101 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1227/2471] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:36:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:36:53.911951 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d71f2b0-254e-4dc4-bc9e-038832d6b940 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:53.912299 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d71f2b0-254e-4dc4-bc9e-038832d6b940 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:53.912587 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d71f2b0-254e-4dc4-bc9e-038832d6b940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:53.936971 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-5d71f2b0-254e-4dc4-bc9e-038832d6b940 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:36:53.938223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-5d71f2b0-254e-4dc4-bc9e-038832d6b940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:36:53.948873 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1245/2472] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:36:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:36:53.951157 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af270b2-8b68-4d3f-a7a0-1c7f2a850cfd None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:36:53.963959 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af270b2-8b68-4d3f-a7a0-1c7f2a850cfd None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:36:53.968660 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a075a3b3-fcdf-4e9f-b693-2e8b2b6d7dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:36:53.969319 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a075a3b3-fcdf-4e9f-b693-2e8b2b6d7dd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:53.969600 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a075a3b3-fcdf-4e9f-b693-2e8b2b6d7dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:53.969878 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a075a3b3-fcdf-4e9f-b693-2e8b2b6d7dd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:53.970637 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a075a3b3-fcdf-4e9f-b693-2e8b2b6d7dd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:36:53.970945 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a075a3b3-fcdf-4e9f-b693-2e8b2b6d7dd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:36:53.972609 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a075a3b3-fcdf-4e9f-b693-2e8b2b6d7dd0 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:36:53.985584 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1228/2473] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:36:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:36:53.989946 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d71f2b0-254e-4dc4-bc9e-038832d6b940 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:36:54.000325 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d71f2b0-254e-4dc4-bc9e-038832d6b940 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:36:54.005247 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9568568e-de5d-44e8-a35c-99f70b03a968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:36:54.005929 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568568e-de5d-44e8-a35c-99f70b03a968 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:54.006218 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568568e-de5d-44e8-a35c-99f70b03a968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:54.006500 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568568e-de5d-44e8-a35c-99f70b03a968 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:54.007328 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9568568e-de5d-44e8-a35c-99f70b03a968 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:36:54.009249 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9568568e-de5d-44e8-a35c-99f70b03a968 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:36:54.111308 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1246/2474] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:36:53 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:36:54.115587 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a075a3b3-fcdf-4e9f-b693-2e8b2b6d7dd0 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:36:54.126401 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a075a3b3-fcdf-4e9f-b693-2e8b2b6d7dd0 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:36:54.137152 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:36:54.190334 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d70eaf-0a6b-43ea-a8f1-3efe6ea89e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:36:54.190963 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d70eaf-0a6b-43ea-a8f1-3efe6ea89e77 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:54.191250 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d70eaf-0a6b-43ea-a8f1-3efe6ea89e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:54.191533 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d70eaf-0a6b-43ea-a8f1-3efe6ea89e77 admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:54.192249 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d70eaf-0a6b-43ea-a8f1-3efe6ea89e77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:36:54.194172 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d70eaf-0a6b-43ea-a8f1-3efe6ea89e77 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:36:54.302410 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1229/2475] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:36:54 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:36:54.308884 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9568568e-de5d-44e8-a35c-99f70b03a968 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:36:54.321939 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9568568e-de5d-44e8-a35c-99f70b03a968 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:36:54.326714 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17de8a62-c123-4e06-88c8-d07686e03a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:36:54.330648 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17de8a62-c123-4e06-88c8-d07686e03a58 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:54.330648 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17de8a62-c123-4e06-88c8-d07686e03a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:54.330648 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17de8a62-c123-4e06-88c8-d07686e03a58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:54.330648 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17de8a62-c123-4e06-88c8-d07686e03a58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:36:54.330648 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17de8a62-c123-4e06-88c8-d07686e03a58 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:36:54.342183 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1247/2476] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:36:54.346335 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d70eaf-0a6b-43ea-a8f1-3efe6ea89e77 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:36:54.357506 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d70eaf-0a6b-43ea-a8f1-3efe6ea89e77 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:36:54.362851 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2be60-2ab0-4ab0-b741-992f8a430610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:36:54.363387 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2be60-2ab0-4ab0-b741-992f8a430610 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:54.363683 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2be60-2ab0-4ab0-b741-992f8a430610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:54.364125 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2be60-2ab0-4ab0-b741-992f8a430610 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:54.364738 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee2be60-2ab0-4ab0-b741-992f8a430610 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:36:54.366323 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee2be60-2ab0-4ab0-b741-992f8a430610 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:36:54.377127 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1230/2477] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:36:54.381308 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17de8a62-c123-4e06-88c8-d07686e03a58 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:36:54.394060 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17de8a62-c123-4e06-88c8-d07686e03a58 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:36:54.399202 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99eee1-13bc-4dfd-9503-7b9ab16160e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:36:54.399890 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99eee1-13bc-4dfd-9503-7b9ab16160e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:54.400205 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99eee1-13bc-4dfd-9503-7b9ab16160e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:54.400488 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99eee1-13bc-4dfd-9503-7b9ab16160e8 admin admin] PATH_INFO: `/v3/projects/5a1e917d8dfe4aee9d739d4df1b268fc/users/561df7f148964b27978333ef1422b9bf/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:54.417489 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd99eee1-13bc-4dfd-9503-7b9ab16160e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a1e917d8dfe4aee9d739d4df1b268fc, user_id=561df7f148964b27978333ef1422b9bf, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:36:54.420296 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd99eee1-13bc-4dfd-9503-7b9ab16160e8 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:36:54.496246 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1248/2478] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:36:54 2022] PUT /identity/v3/projects/5a1e917d8dfe4aee9d739d4df1b268fc/users/561df7f148964b27978333ef1422b9bf/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:36:54.508312 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8201c0-9dd7-4d79-b8cc-e947b594c593 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:54.508687 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8201c0-9dd7-4d79-b8cc-e947b594c593 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:54.508978 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8201c0-9dd7-4d79-b8cc-e947b594c593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:54.536957 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-bd8201c0-9dd7-4d79-b8cc-e947b594c593 None None] MFA Rules not processed for user `561df7f148964b27978333ef1422b9bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:36:54.538178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-bd8201c0-9dd7-4d79-b8cc-e947b594c593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:36:54.601084 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1231/2479] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:36:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:36:54.611959 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99eee1-13bc-4dfd-9503-7b9ab16160e8 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:36:54.614338 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-cd99eee1-13bc-4dfd-9503-7b9ab16160e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:36:54.696722 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99eee1-13bc-4dfd-9503-7b9ab16160e8 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:36:54.702986 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee618623-5a65-4da8-bc36-e1605aa24d46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:36:54.703771 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee618623-5a65-4da8-bc36-e1605aa24d46 service nova] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:36:54.704060 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee618623-5a65-4da8-bc36-e1605aa24d46 service nova] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:36:54.704358 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee618623-5a65-4da8-bc36-e1605aa24d46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:36:54.714596 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee618623-5a65-4da8-bc36-e1605aa24d46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:36:54.716711 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee618623-5a65-4da8-bc36-e1605aa24d46 service nova] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:36:54.727458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1249/2480] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:36:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:37:03.785334 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8201c0-9dd7-4d79-b8cc-e947b594c593 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:03.850636 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8201c0-9dd7-4d79-b8cc-e947b594c593 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:03.855310 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309e09d-9eac-4348-8dcb-08681829b22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:03.856265 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309e09d-9eac-4348-8dcb-08681829b22a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:03.856550 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309e09d-9eac-4348-8dcb-08681829b22a admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:03.856828 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309e09d-9eac-4348-8dcb-08681829b22a admin admin] PATH_INFO: `/v3/users/561df7f148964b27978333ef1422b9bf` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:03.857773 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c2ac4-8b07-4fd7-8915-f0acd4fca040 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:03.858219 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f309e09d-9eac-4348-8dcb-08681829b22a admin admin] RBAC: Authorizing `identity:delete_user(user_id=561df7f148964b27978333ef1422b9bf)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:03.858313 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c2ac4-8b07-4fd7-8915-f0acd4fca040 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:03.858528 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c2ac4-8b07-4fd7-8915-f0acd4fca040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:03.859934 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f309e09d-9eac-4348-8dcb-08681829b22a admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:03.878198 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-4c3c2ac4-8b07-4fd7-8915-f0acd4fca040 None None] MFA Rules not processed for user `1654fb6a81e14d6b99ab495bb6d029d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:37:03.879414 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-4c3c2ac4-8b07-4fd7-8915-f0acd4fca040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:03.886271 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:37:03.934531 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1250/2481] 10.222.0.40 () {58 vars in 967 bytes} [Fri Aug 5 15:37:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:37:03.949332 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c2ac4-8b07-4fd7-8915-f0acd4fca040 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:03.949332 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-4c3c2ac4-8b07-4fd7-8915-f0acd4fca040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:03.966334 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:37:03.990756 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-f309e09d-9eac-4348-8dcb-08681829b22a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '561df7f148964b27978333ef1422b9bf'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:03.998528 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-f309e09d-9eac-4348-8dcb-08681829b22a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '561df7f148964b27978333ef1422b9bf'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:03.999218 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-f309e09d-9eac-4348-8dcb-08681829b22a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '561df7f148964b27978333ef1422b9bf'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:04.011803 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-f309e09d-9eac-4348-8dcb-08681829b22a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '561df7f148964b27978333ef1422b9bf'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:04.012478 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-f309e09d-9eac-4348-8dcb-08681829b22a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '561df7f148964b27978333ef1422b9bf'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:04.020611 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c2ac4-8b07-4fd7-8915-f0acd4fca040 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:04.025828 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb51ecf-efb8-4778-90d9-7fc64f9d9ed7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85154ce7446245738edcc8702dbf6861', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:04.026447 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb51ecf-efb8-4778-90d9-7fc64f9d9ed7 service swift] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:04.026730 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb51ecf-efb8-4778-90d9-7fc64f9d9ed7 service swift] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:04.027047 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb51ecf-efb8-4778-90d9-7fc64f9d9ed7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:04.028971 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-4eb51ecf-efb8-4778-90d9-7fc64f9d9ed7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:04.039649 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb51ecf-efb8-4778-90d9-7fc64f9d9ed7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:04.041306 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb51ecf-efb8-4778-90d9-7fc64f9d9ed7 service swift] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:04.049198 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1251/2482] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 5 15:37:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 05 15:37:04.100645 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1232/2483] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:37:03 2022] DELETE /identity/v3/users/561df7f148964b27978333ef1422b9bf => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:37:04.107172 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8972c0b-ed9f-4b40-9093-598bf30c6e5d None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:04.107172 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8972c0b-ed9f-4b40-9093-598bf30c6e5d None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:04.107172 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8972c0b-ed9f-4b40-9093-598bf30c6e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:04.127382 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-a8972c0b-ed9f-4b40-9093-598bf30c6e5d None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:37:04.129148 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-a8972c0b-ed9f-4b40-9093-598bf30c6e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:04.208572 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1252/2484] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:37:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:37:04.218648 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309e09d-9eac-4348-8dcb-08681829b22a admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:04.220160 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-f309e09d-9eac-4348-8dcb-08681829b22a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:04.294746 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309e09d-9eac-4348-8dcb-08681829b22a admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:04.298377 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6010cc-0dcb-4bf1-b81c-53a790a4112d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:04.299000 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6010cc-0dcb-4bf1-b81c-53a790a4112d service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:04.299442 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6010cc-0dcb-4bf1-b81c-53a790a4112d service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:04.299697 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6010cc-0dcb-4bf1-b81c-53a790a4112d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:04.310699 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6010cc-0dcb-4bf1-b81c-53a790a4112d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:04.312446 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6010cc-0dcb-4bf1-b81c-53a790a4112d service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:04.323085 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1233/2485] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:37:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:37:04.853559 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8972c0b-ed9f-4b40-9093-598bf30c6e5d None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:04.857797 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-a8972c0b-ed9f-4b40-9093-598bf30c6e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:04.875402 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8972c0b-ed9f-4b40-9093-598bf30c6e5d None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:04.881383 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b770f4c5-a274-42f9-887f-d9a6c6401b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:04.882293 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770f4c5-a274-42f9-887f-d9a6c6401b50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:04.882518 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770f4c5-a274-42f9-887f-d9a6c6401b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:04.882746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770f4c5-a274-42f9-887f-d9a6c6401b50 admin admin] PATH_INFO: `/v3/projects/5a1e917d8dfe4aee9d739d4df1b268fc` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:04.884029 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b770f4c5-a274-42f9-887f-d9a6c6401b50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a1e917d8dfe4aee9d739d4df1b268fc)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:04.886281 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b770f4c5-a274-42f9-887f-d9a6c6401b50 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:04.996365 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-b770f4c5-a274-42f9-887f-d9a6c6401b50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a1e917d8dfe4aee9d739d4df1b268fc', 'request_id': 'req-b770f4c5-a274-42f9-887f-d9a6c6401b50'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:05.002800 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-b770f4c5-a274-42f9-887f-d9a6c6401b50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a1e917d8dfe4aee9d739d4df1b268fc', 'request_id': 'req-b770f4c5-a274-42f9-887f-d9a6c6401b50'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:05.009477 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-b770f4c5-a274-42f9-887f-d9a6c6401b50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a1e917d8dfe4aee9d739d4df1b268fc', 'request_id': 'req-b770f4c5-a274-42f9-887f-d9a6c6401b50'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:05.108209 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-b770f4c5-a274-42f9-887f-d9a6c6401b50 admin admin] The token cache is being invalidate because project 5a1e917d8dfe4aee9d739d4df1b268fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:37:05.108506 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-b770f4c5-a274-42f9-887f-d9a6c6401b50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:05.109174 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-b770f4c5-a274-42f9-887f-d9a6c6401b50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:05.110716 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1253/2486] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:37:04 2022] DELETE /identity/v3/projects/5a1e917d8dfe4aee9d739d4df1b268fc => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:37:05.116848 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60165771-a99f-48b6-8cb4-d1b6681c00c6 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:05.117318 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60165771-a99f-48b6-8cb4-d1b6681c00c6 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:05.117410 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60165771-a99f-48b6-8cb4-d1b6681c00c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:05.138448 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-60165771-a99f-48b6-8cb4-d1b6681c00c6 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:37:05.139672 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-60165771-a99f-48b6-8cb4-d1b6681c00c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:05.208841 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1234/2487] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:37:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:37:05.217318 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd645ac7-49d6-421b-8be4-3af2d2d33811 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:05.217717 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd645ac7-49d6-421b-8be4-3af2d2d33811 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:05.218088 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd645ac7-49d6-421b-8be4-3af2d2d33811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:05.238302 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-dd645ac7-49d6-421b-8be4-3af2d2d33811 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:37:05.239497 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-dd645ac7-49d6-421b-8be4-3af2d2d33811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:05.247424 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1254/2488] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:37:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:37:05.251716 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60165771-a99f-48b6-8cb4-d1b6681c00c6 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:05.262209 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60165771-a99f-48b6-8cb4-d1b6681c00c6 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:05.266820 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e6347-4397-4229-9ed8-2b78db6863cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:05.267437 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e6347-4397-4229-9ed8-2b78db6863cd admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:05.267718 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e6347-4397-4229-9ed8-2b78db6863cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:05.267999 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e6347-4397-4229-9ed8-2b78db6863cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:05.268995 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e6347-4397-4229-9ed8-2b78db6863cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:37:05.269306 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e6347-4397-4229-9ed8-2b78db6863cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:05.270939 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e6347-4397-4229-9ed8-2b78db6863cd admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:05.283879 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1235/2489] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:37:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:37:05.288053 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd645ac7-49d6-421b-8be4-3af2d2d33811 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:05.298416 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd645ac7-49d6-421b-8be4-3af2d2d33811 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:05.303104 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03548e7-d4f5-49bb-a719-dff8ff9546f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:05.303708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03548e7-d4f5-49bb-a719-dff8ff9546f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:05.303996 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03548e7-d4f5-49bb-a719-dff8ff9546f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:05.304316 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03548e7-d4f5-49bb-a719-dff8ff9546f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:05.304985 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03548e7-d4f5-49bb-a719-dff8ff9546f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:05.306954 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03548e7-d4f5-49bb-a719-dff8ff9546f6 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:05.400553 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:37:05.411073 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1255/2490] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:37:05 2022] POST /identity/v3/projects => generated 377 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:37:05.414637 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e6347-4397-4229-9ed8-2b78db6863cd admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:05.423131 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e6347-4397-4229-9ed8-2b78db6863cd admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:05.452446 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:37:05.485874 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05af732-4ff7-490e-8370-5eba257681b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:05.485874 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05af732-4ff7-490e-8370-5eba257681b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:05.485874 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05af732-4ff7-490e-8370-5eba257681b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:05.485874 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05af732-4ff7-490e-8370-5eba257681b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:05.486258 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05af732-4ff7-490e-8370-5eba257681b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:05.486258 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05af732-4ff7-490e-8370-5eba257681b7 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:05.588494 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1236/2491] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:37:05 2022] POST /identity/v3/users => generated 289 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:37:05.589272 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03548e7-d4f5-49bb-a719-dff8ff9546f6 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:05.600398 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03548e7-d4f5-49bb-a719-dff8ff9546f6 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:05.606639 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380fa6a-1592-4bc2-811f-77528329c9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:05.611226 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380fa6a-1592-4bc2-811f-77528329c9f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:05.611226 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380fa6a-1592-4bc2-811f-77528329c9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:05.611226 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380fa6a-1592-4bc2-811f-77528329c9f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:05.611226 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380fa6a-1592-4bc2-811f-77528329c9f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:05.611226 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380fa6a-1592-4bc2-811f-77528329c9f5 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:05.621028 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1256/2492] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:37:05.625272 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05af732-4ff7-490e-8370-5eba257681b7 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:05.636332 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05af732-4ff7-490e-8370-5eba257681b7 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:05.644032 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac621981-7c2e-4d91-8e2b-581713247129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:05.644608 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac621981-7c2e-4d91-8e2b-581713247129 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:05.644895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac621981-7c2e-4d91-8e2b-581713247129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:05.645173 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac621981-7c2e-4d91-8e2b-581713247129 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:05.645868 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac621981-7c2e-4d91-8e2b-581713247129 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:05.647546 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac621981-7c2e-4d91-8e2b-581713247129 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:05.659157 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1237/2493] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:37:05.666301 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380fa6a-1592-4bc2-811f-77528329c9f5 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:05.676564 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380fa6a-1592-4bc2-811f-77528329c9f5 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:05.681370 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d13bf2-c3fb-4dad-bd5d-f64e37db7f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:05.682037 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d13bf2-c3fb-4dad-bd5d-f64e37db7f15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:05.682328 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d13bf2-c3fb-4dad-bd5d-f64e37db7f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:05.682614 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d13bf2-c3fb-4dad-bd5d-f64e37db7f15 admin admin] PATH_INFO: `/v3/projects/458fa0512e1c42aba76f1a6a05e13721/users/15d9bdcececb498293b5987ba9bcd60f/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:05.698580 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d13bf2-c3fb-4dad-bd5d-f64e37db7f15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=458fa0512e1c42aba76f1a6a05e13721, user_id=15d9bdcececb498293b5987ba9bcd60f, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:05.701344 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d13bf2-c3fb-4dad-bd5d-f64e37db7f15 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:05.787691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1257/2494] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:37:05 2022] PUT /identity/v3/projects/458fa0512e1c42aba76f1a6a05e13721/users/15d9bdcececb498293b5987ba9bcd60f/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:37:05.801100 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2fe07a-e4fc-495b-96b4-fd11c4e6c47d None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:05.801492 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2fe07a-e4fc-495b-96b4-fd11c4e6c47d None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:05.801775 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2fe07a-e4fc-495b-96b4-fd11c4e6c47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:05.829537 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-5d2fe07a-e4fc-495b-96b4-fd11c4e6c47d None None] MFA Rules not processed for user `15d9bdcececb498293b5987ba9bcd60f`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:37:05.830755 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-5d2fe07a-e4fc-495b-96b4-fd11c4e6c47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:05.896196 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1238/2495] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:37:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:37:05.901208 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d13bf2-c3fb-4dad-bd5d-f64e37db7f15 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:05.962000 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d13bf2-c3fb-4dad-bd5d-f64e37db7f15 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:05.967307 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71114796-55b9-409c-b2e0-330aa5122866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:05.967917 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71114796-55b9-409c-b2e0-330aa5122866 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:05.968241 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71114796-55b9-409c-b2e0-330aa5122866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:05.968534 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71114796-55b9-409c-b2e0-330aa5122866 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:05.969195 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71114796-55b9-409c-b2e0-330aa5122866 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:05.971114 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71114796-55b9-409c-b2e0-330aa5122866 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:06.046164 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1258/2496] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:37:05 2022] POST /identity/v3/projects => generated 377 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:37:06.050442 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2fe07a-e4fc-495b-96b4-fd11c4e6c47d None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:06.061587 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2fe07a-e4fc-495b-96b4-fd11c4e6c47d None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:06.116672 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5fc9f2-7d23-402c-b6e9-0293b0a8aca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:06.117284 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5fc9f2-7d23-402c-b6e9-0293b0a8aca7 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:06.117815 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5fc9f2-7d23-402c-b6e9-0293b0a8aca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:06.117815 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5fc9f2-7d23-402c-b6e9-0293b0a8aca7 admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:06.118984 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5fc9f2-7d23-402c-b6e9-0293b0a8aca7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:06.120926 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5fc9f2-7d23-402c-b6e9-0293b0a8aca7 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:06.200569 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1239/2497] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:37:06 2022] POST /identity/v3/users => generated 295 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:37:06.204549 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71114796-55b9-409c-b2e0-330aa5122866 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:06.214845 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71114796-55b9-409c-b2e0-330aa5122866 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:06.219674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56681817-3a78-47a9-b87b-10358938a615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:06.220421 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56681817-3a78-47a9-b87b-10358938a615 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:06.220713 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56681817-3a78-47a9-b87b-10358938a615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:06.221032 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56681817-3a78-47a9-b87b-10358938a615 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:06.221732 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56681817-3a78-47a9-b87b-10358938a615 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:06.223720 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56681817-3a78-47a9-b87b-10358938a615 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:06.239372 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1259/2498] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:37:06.240690 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5fc9f2-7d23-402c-b6e9-0293b0a8aca7 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:06.254100 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5fc9f2-7d23-402c-b6e9-0293b0a8aca7 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:06.258755 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86782640-f13f-4a33-8fbf-496634ebe5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:06.260089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86782640-f13f-4a33-8fbf-496634ebe5d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:06.260472 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86782640-f13f-4a33-8fbf-496634ebe5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:06.260765 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86782640-f13f-4a33-8fbf-496634ebe5d6 admin admin] PATH_INFO: `/v3/projects/c8571d46c4fb4aad8214fb33de4b9658/users/d9e2adfb84244f0082d4f74fa38c1ead/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:06.275690 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86782640-f13f-4a33-8fbf-496634ebe5d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8571d46c4fb4aad8214fb33de4b9658, user_id=d9e2adfb84244f0082d4f74fa38c1ead, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:06.278403 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86782640-f13f-4a33-8fbf-496634ebe5d6 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:06.348498 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1240/2499] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:37:06 2022] PUT /identity/v3/projects/c8571d46c4fb4aad8214fb33de4b9658/users/d9e2adfb84244f0082d4f74fa38c1ead/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:37:06.352566 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56681817-3a78-47a9-b87b-10358938a615 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:06.363306 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56681817-3a78-47a9-b87b-10358938a615 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:06.420972 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eff611-281c-4e15-aaf0-c19bfbc1bef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:06.421517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eff611-281c-4e15-aaf0-c19bfbc1bef4 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:06.421809 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eff611-281c-4e15-aaf0-c19bfbc1bef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:06.422117 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eff611-281c-4e15-aaf0-c19bfbc1bef4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:06.422792 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eff611-281c-4e15-aaf0-c19bfbc1bef4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:06.424505 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eff611-281c-4e15-aaf0-c19bfbc1bef4 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:06.438443 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1260/2500] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:37:06.439555 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86782640-f13f-4a33-8fbf-496634ebe5d6 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:06.446647 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:37:06.450153 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86782640-f13f-4a33-8fbf-496634ebe5d6 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:06.454984 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd2b596-8f88-4538-9e93-545bcf82ba8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:06.455644 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2b596-8f88-4538-9e93-545bcf82ba8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:06.455930 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2b596-8f88-4538-9e93-545bcf82ba8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:06.456239 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2b596-8f88-4538-9e93-545bcf82ba8d admin admin] PATH_INFO: `/v3/projects/c8571d46c4fb4aad8214fb33de4b9658/users/d9e2adfb84244f0082d4f74fa38c1ead/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:06.458215 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd2b596-8f88-4538-9e93-545bcf82ba8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8571d46c4fb4aad8214fb33de4b9658, user_id=d9e2adfb84244f0082d4f74fa38c1ead, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:06.460896 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd2b596-8f88-4538-9e93-545bcf82ba8d admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:06.478376 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1241/2501] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:37:06 2022] PUT /identity/v3/projects/c8571d46c4fb4aad8214fb33de4b9658/users/d9e2adfb84244f0082d4f74fa38c1ead/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:37:06.488804 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af750f4-cebe-4c20-94e3-466a8e502c38 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:06.489096 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af750f4-cebe-4c20-94e3-466a8e502c38 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:06.489450 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af750f4-cebe-4c20-94e3-466a8e502c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:06.519360 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-1af750f4-cebe-4c20-94e3-466a8e502c38 None None] MFA Rules not processed for user `d9e2adfb84244f0082d4f74fa38c1ead`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:37:06.520932 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-1af750f4-cebe-4c20-94e3-466a8e502c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:06.547907 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:37:06.603694 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1261/2502] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:37:06 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:37:06.614649 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd2b596-8f88-4538-9e93-545bcf82ba8d admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:06.621103 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-8bd2b596-8f88-4538-9e93-545bcf82ba8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:06.680238 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd2b596-8f88-4538-9e93-545bcf82ba8d admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:06.684619 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e6b10-21dd-466b-80aa-a5347069ffd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:06.690341 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e6b10-21dd-466b-80aa-a5347069ffd4 service cinder] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:06.690341 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e6b10-21dd-466b-80aa-a5347069ffd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:06.690341 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e6b10-21dd-466b-80aa-a5347069ffd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:06.740922 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7e6b10-21dd-466b-80aa-a5347069ffd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:06.742344 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7e6b10-21dd-466b-80aa-a5347069ffd4 service cinder] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:06.751091 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1242/2503] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:37:06 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:37:12.014637 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af750f4-cebe-4c20-94e3-466a8e502c38 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:12.016287 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-1af750f4-cebe-4c20-94e3-466a8e502c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:12.031657 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af750f4-cebe-4c20-94e3-466a8e502c38 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:12.035391 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d67c5-de42-45ea-a220-9de35c834fc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:12.036010 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d67c5-de42-45ea-a220-9de35c834fc3 service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:12.036348 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d67c5-de42-45ea-a220-9de35c834fc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:12.036628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d67c5-de42-45ea-a220-9de35c834fc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:12.047933 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033d67c5-de42-45ea-a220-9de35c834fc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:12.049418 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033d67c5-de42-45ea-a220-9de35c834fc3 service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:12.059851 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1262/2504] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:37:12 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:37:16.747242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=51,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:37:22.054701 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=25 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:37:25.303981 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce854249-731e-4132-a4e2-35af1af9040b None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:25.304523 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce854249-731e-4132-a4e2-35af1af9040b None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:25.304624 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce854249-731e-4132-a4e2-35af1af9040b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:25.324051 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-ce854249-731e-4132-a4e2-35af1af9040b None None] MFA Rules not processed for user `1654fb6a81e14d6b99ab495bb6d029d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:37:25.325326 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-ce854249-731e-4132-a4e2-35af1af9040b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:25.383098 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1243/2505] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 5 15:37:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:37:25.405791 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d67c5-de42-45ea-a220-9de35c834fc3 service cinder] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:25.407479 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-033d67c5-de42-45ea-a220-9de35c834fc3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:25.476964 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d67c5-de42-45ea-a220-9de35c834fc3 service cinder] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:25.482293 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e60392-67b2-4e11-892a-a2f059d7c0e8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85154ce7446245738edcc8702dbf6861', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:25.482910 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e60392-67b2-4e11-892a-a2f059d7c0e8 service swift] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:25.483283 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e60392-67b2-4e11-892a-a2f059d7c0e8 service swift] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:25.483562 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e60392-67b2-4e11-892a-a2f059d7c0e8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:25.492933 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e60392-67b2-4e11-892a-a2f059d7c0e8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:25.494581 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e60392-67b2-4e11-892a-a2f059d7c0e8 service swift] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:25.503775 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1263/2506] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 5 15:37:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 15:37:35.375359 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=25 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:37:35.500490 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=32 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:37:41.367211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce854249-731e-4132-a4e2-35af1af9040b None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:41.428596 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce854249-731e-4132-a4e2-35af1af9040b None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:41.433006 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb883b2-402e-4430-b40d-56243a7d1bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:41.434204 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb883b2-402e-4430-b40d-56243a7d1bd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:41.434489 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb883b2-402e-4430-b40d-56243a7d1bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:41.434758 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb883b2-402e-4430-b40d-56243a7d1bd3 admin admin] PATH_INFO: `/v3/users/15d9bdcececb498293b5987ba9bcd60f` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:41.435910 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb883b2-402e-4430-b40d-56243a7d1bd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15d9bdcececb498293b5987ba9bcd60f)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:41.437843 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb883b2-402e-4430-b40d-56243a7d1bd3 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:41.637332 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-5eb883b2-402e-4430-b40d-56243a7d1bd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15d9bdcececb498293b5987ba9bcd60f'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:41.644223 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-5eb883b2-402e-4430-b40d-56243a7d1bd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15d9bdcececb498293b5987ba9bcd60f'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:41.644919 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-5eb883b2-402e-4430-b40d-56243a7d1bd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15d9bdcececb498293b5987ba9bcd60f'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:41.657268 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-5eb883b2-402e-4430-b40d-56243a7d1bd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15d9bdcececb498293b5987ba9bcd60f'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:41.657945 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-5eb883b2-402e-4430-b40d-56243a7d1bd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15d9bdcececb498293b5987ba9bcd60f'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:41.732553 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1244/2507] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:37:41 2022] DELETE /identity/v3/users/15d9bdcececb498293b5987ba9bcd60f => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:37:41.737078 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b5342-b379-4832-8cc9-597c496ea80e None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:41.737550 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b5342-b379-4832-8cc9-597c496ea80e None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:41.737655 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b5342-b379-4832-8cc9-597c496ea80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:41.756948 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-2e7b5342-b379-4832-8cc9-597c496ea80e None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:37:41.758149 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-2e7b5342-b379-4832-8cc9-597c496ea80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:41.818266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1264/2508] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:37:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:37:41.828124 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb883b2-402e-4430-b40d-56243a7d1bd3 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:41.829575 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-5eb883b2-402e-4430-b40d-56243a7d1bd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:41.891807 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb883b2-402e-4430-b40d-56243a7d1bd3 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:41.895498 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4a8bda-482a-4ce7-b8f2-5a92262f7d24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:41.896416 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a8bda-482a-4ce7-b8f2-5a92262f7d24 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:41.896702 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a8bda-482a-4ce7-b8f2-5a92262f7d24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:41.896979 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a8bda-482a-4ce7-b8f2-5a92262f7d24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:41.906544 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4a8bda-482a-4ce7-b8f2-5a92262f7d24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:41.907661 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4a8bda-482a-4ce7-b8f2-5a92262f7d24 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:41.918416 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1245/2509] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:37:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:37:41.981110 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c886844-3421-4011-88d0-306afec7dc47 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:41.985790 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c886844-3421-4011-88d0-306afec7dc47 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:41.985790 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c886844-3421-4011-88d0-306afec7dc47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:42.000928 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-8c886844-3421-4011-88d0-306afec7dc47 None None] MFA Rules not processed for user `1654fb6a81e14d6b99ab495bb6d029d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:37:42.003034 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-8c886844-3421-4011-88d0-306afec7dc47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:42.058856 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1265/2510] 10.222.0.40 () {58 vars in 967 bytes} [Fri Aug 5 15:37:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:37:42.071566 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4a8bda-482a-4ce7-b8f2-5a92262f7d24 service neutron] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:42.073009 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-1f4a8bda-482a-4ce7-b8f2-5a92262f7d24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:42.100121 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:37:42.154180 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4a8bda-482a-4ce7-b8f2-5a92262f7d24 service neutron] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:42.161336 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffd8969-3869-4723-9346-5d77137087d2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85154ce7446245738edcc8702dbf6861', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:42.161978 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd8969-3869-4723-9346-5d77137087d2 service swift] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:42.162258 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd8969-3869-4723-9346-5d77137087d2 service swift] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:42.162539 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd8969-3869-4723-9346-5d77137087d2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:42.174629 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffd8969-3869-4723-9346-5d77137087d2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:42.174966 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffd8969-3869-4723-9346-5d77137087d2 service swift] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:42.184815 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1246/2511] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 5 15:37:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 15:37:42.288497 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c886844-3421-4011-88d0-306afec7dc47 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:42.289939 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-8c886844-3421-4011-88d0-306afec7dc47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:42.313059 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c886844-3421-4011-88d0-306afec7dc47 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:42.318313 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a17b5fa-7203-4a55-af8d-e18148b0128a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:42.318986 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a17b5fa-7203-4a55-af8d-e18148b0128a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:42.319273 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a17b5fa-7203-4a55-af8d-e18148b0128a admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:42.319552 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a17b5fa-7203-4a55-af8d-e18148b0128a admin admin] PATH_INFO: `/v3/projects/458fa0512e1c42aba76f1a6a05e13721` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:42.324891 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a17b5fa-7203-4a55-af8d-e18148b0128a admin admin] RBAC: Authorizing `identity:delete_project(project_id=458fa0512e1c42aba76f1a6a05e13721)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:42.326986 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a17b5fa-7203-4a55-af8d-e18148b0128a admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:42.455046 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-6a17b5fa-7203-4a55-af8d-e18148b0128a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '458fa0512e1c42aba76f1a6a05e13721', 'request_id': 'req-6a17b5fa-7203-4a55-af8d-e18148b0128a'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:42.461660 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-6a17b5fa-7203-4a55-af8d-e18148b0128a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '458fa0512e1c42aba76f1a6a05e13721', 'request_id': 'req-6a17b5fa-7203-4a55-af8d-e18148b0128a'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:42.468119 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-6a17b5fa-7203-4a55-af8d-e18148b0128a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '458fa0512e1c42aba76f1a6a05e13721', 'request_id': 'req-6a17b5fa-7203-4a55-af8d-e18148b0128a'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:42.538512 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-6a17b5fa-7203-4a55-af8d-e18148b0128a admin admin] The token cache is being invalidate because project 458fa0512e1c42aba76f1a6a05e13721 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:37:42.538916 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-6a17b5fa-7203-4a55-af8d-e18148b0128a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:42.539475 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-6a17b5fa-7203-4a55-af8d-e18148b0128a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:42.541198 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1266/2512] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:37:42 2022] DELETE /identity/v3/projects/458fa0512e1c42aba76f1a6a05e13721 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:37:42.545055 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffd8969-3869-4723-9346-5d77137087d2 service swift] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:42.555896 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffd8969-3869-4723-9346-5d77137087d2 service swift] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:42.612324 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305914af-96bd-45d0-96a7-e7f8cf0f95b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:42.612956 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305914af-96bd-45d0-96a7-e7f8cf0f95b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:42.613245 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305914af-96bd-45d0-96a7-e7f8cf0f95b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:42.613527 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305914af-96bd-45d0-96a7-e7f8cf0f95b4 admin admin] PATH_INFO: `/v3/users/d9e2adfb84244f0082d4f74fa38c1ead` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:42.614693 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305914af-96bd-45d0-96a7-e7f8cf0f95b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9e2adfb84244f0082d4f74fa38c1ead)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:42.616639 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305914af-96bd-45d0-96a7-e7f8cf0f95b4 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:42.850195 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-305914af-96bd-45d0-96a7-e7f8cf0f95b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9e2adfb84244f0082d4f74fa38c1ead'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:42.857697 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-305914af-96bd-45d0-96a7-e7f8cf0f95b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9e2adfb84244f0082d4f74fa38c1ead'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:42.858389 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-305914af-96bd-45d0-96a7-e7f8cf0f95b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9e2adfb84244f0082d4f74fa38c1ead'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:42.870507 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-305914af-96bd-45d0-96a7-e7f8cf0f95b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9e2adfb84244f0082d4f74fa38c1ead'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:42.871657 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-305914af-96bd-45d0-96a7-e7f8cf0f95b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9e2adfb84244f0082d4f74fa38c1ead'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:42.967246 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1247/2513] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:37:42 2022] DELETE /identity/v3/users/d9e2adfb84244f0082d4f74fa38c1ead => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:37:43.363446 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a17b5fa-7203-4a55-af8d-e18148b0128a admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:43.364991 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-6a17b5fa-7203-4a55-af8d-e18148b0128a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:43.379175 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a17b5fa-7203-4a55-af8d-e18148b0128a admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:43.406846 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:37:43.438499 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5183c9-ea26-4827-903c-9d07e493feb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:43.439144 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5183c9-ea26-4827-903c-9d07e493feb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:43.439499 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5183c9-ea26-4827-903c-9d07e493feb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:43.439789 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5183c9-ea26-4827-903c-9d07e493feb7 admin admin] PATH_INFO: `/v3/projects/c8571d46c4fb4aad8214fb33de4b9658` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:43.440985 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5183c9-ea26-4827-903c-9d07e493feb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8571d46c4fb4aad8214fb33de4b9658)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:43.443675 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5183c9-ea26-4827-903c-9d07e493feb7 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:43.562279 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-5a5183c9-ea26-4827-903c-9d07e493feb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8571d46c4fb4aad8214fb33de4b9658', 'request_id': 'req-5a5183c9-ea26-4827-903c-9d07e493feb7'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:43.568379 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-5a5183c9-ea26-4827-903c-9d07e493feb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8571d46c4fb4aad8214fb33de4b9658', 'request_id': 'req-5a5183c9-ea26-4827-903c-9d07e493feb7'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:43.574032 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-5a5183c9-ea26-4827-903c-9d07e493feb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8571d46c4fb4aad8214fb33de4b9658', 'request_id': 'req-5a5183c9-ea26-4827-903c-9d07e493feb7'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:43.652936 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-5a5183c9-ea26-4827-903c-9d07e493feb7 admin admin] The token cache is being invalidate because project c8571d46c4fb4aad8214fb33de4b9658 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:37:43.653324 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-5a5183c9-ea26-4827-903c-9d07e493feb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:43.653951 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-5a5183c9-ea26-4827-903c-9d07e493feb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:43.655460 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1267/2514] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:37:43 2022] DELETE /identity/v3/projects/c8571d46c4fb4aad8214fb33de4b9658 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:37:43.660318 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839fa2f-5b06-4183-9209-b6a43780d3cd None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:43.660929 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839fa2f-5b06-4183-9209-b6a43780d3cd None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:43.660929 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839fa2f-5b06-4183-9209-b6a43780d3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:43.680811 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-b839fa2f-5b06-4183-9209-b6a43780d3cd None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:37:43.682024 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-b839fa2f-5b06-4183-9209-b6a43780d3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:43.708144 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:37:43.741847 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1248/2515] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:37:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:37:43.749726 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05ed2a-a37b-409b-bf8b-a4660165de44 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:43.750022 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05ed2a-a37b-409b-bf8b-a4660165de44 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:43.750308 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05ed2a-a37b-409b-bf8b-a4660165de44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:43.771098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-fc05ed2a-a37b-409b-bf8b-a4660165de44 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:37:43.772622 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-fc05ed2a-a37b-409b-bf8b-a4660165de44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:43.781138 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1268/2516] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:37:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:37:43.785356 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b839fa2f-5b06-4183-9209-b6a43780d3cd None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:43.795627 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b839fa2f-5b06-4183-9209-b6a43780d3cd None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:43.800266 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fabb3e-c9b5-4946-ba10-733a53c65a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:43.800876 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fabb3e-c9b5-4946-ba10-733a53c65a51 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:43.801164 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fabb3e-c9b5-4946-ba10-733a53c65a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:43.801445 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fabb3e-c9b5-4946-ba10-733a53c65a51 admin admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:43.802138 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fabb3e-c9b5-4946-ba10-733a53c65a51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:37:43.802450 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fabb3e-c9b5-4946-ba10-733a53c65a51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:43.804033 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fabb3e-c9b5-4946-ba10-733a53c65a51 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:43.815707 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1249/2517] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:37:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:37:43.819757 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc05ed2a-a37b-409b-bf8b-a4660165de44 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:43.829745 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc05ed2a-a37b-409b-bf8b-a4660165de44 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:43.834551 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d56e3-ba41-4360-8f1e-823c10717242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:43.835160 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d56e3-ba41-4360-8f1e-823c10717242 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:43.835446 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d56e3-ba41-4360-8f1e-823c10717242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:43.835726 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d56e3-ba41-4360-8f1e-823c10717242 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:43.836439 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4d56e3-ba41-4360-8f1e-823c10717242 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:43.838350 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4d56e3-ba41-4360-8f1e-823c10717242 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:43.933895 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1269/2518] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:37:43 2022] POST /identity/v3/projects => generated 385 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:37:43.938189 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fabb3e-c9b5-4946-ba10-733a53c65a51 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:43.948470 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fabb3e-c9b5-4946-ba10-733a53c65a51 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:44.004695 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92442e43-88ca-4747-b87b-fef15ee9f985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:44.005357 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92442e43-88ca-4747-b87b-fef15ee9f985 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:44.005646 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92442e43-88ca-4747-b87b-fef15ee9f985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:44.005927 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92442e43-88ca-4747-b87b-fef15ee9f985 admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:44.006608 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92442e43-88ca-4747-b87b-fef15ee9f985 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:44.008572 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92442e43-88ca-4747-b87b-fef15ee9f985 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:44.110093 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1250/2519] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:37:43 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:37:44.114282 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d56e3-ba41-4360-8f1e-823c10717242 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:44.124811 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d56e3-ba41-4360-8f1e-823c10717242 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:44.129521 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68df681-495d-4838-8551-248bc9656066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:44.130133 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68df681-495d-4838-8551-248bc9656066 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:44.130423 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68df681-495d-4838-8551-248bc9656066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:44.130708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68df681-495d-4838-8551-248bc9656066 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:44.131432 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68df681-495d-4838-8551-248bc9656066 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:44.133072 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68df681-495d-4838-8551-248bc9656066 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:44.143940 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1270/2520] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:37:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:37:44.148679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92442e43-88ca-4747-b87b-fef15ee9f985 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:44.159329 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92442e43-88ca-4747-b87b-fef15ee9f985 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:44.164020 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec3eab-fe5b-4086-bd6e-529b4676a5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:44.164652 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec3eab-fe5b-4086-bd6e-529b4676a5ec admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:44.164948 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec3eab-fe5b-4086-bd6e-529b4676a5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:44.165214 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec3eab-fe5b-4086-bd6e-529b4676a5ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:44.165910 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feec3eab-fe5b-4086-bd6e-529b4676a5ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:44.167496 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feec3eab-fe5b-4086-bd6e-529b4676a5ec admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:44.178454 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1251/2521] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:37:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:37:44.182594 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68df681-495d-4838-8551-248bc9656066 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:44.193152 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68df681-495d-4838-8551-248bc9656066 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:44.197824 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faef38e3-cefd-4e10-8f8b-5e185af911ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:44.198480 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef38e3-cefd-4e10-8f8b-5e185af911ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:44.198770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef38e3-cefd-4e10-8f8b-5e185af911ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:44.199704 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef38e3-cefd-4e10-8f8b-5e185af911ff admin admin] PATH_INFO: `/v3/projects/b0f4625558bf47098a59298065f9765b/users/51ca5c1e25d045fa9f132841533616be/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:44.214141 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faef38e3-cefd-4e10-8f8b-5e185af911ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0f4625558bf47098a59298065f9765b, user_id=51ca5c1e25d045fa9f132841533616be, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:44.216820 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faef38e3-cefd-4e10-8f8b-5e185af911ff admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:44.275698 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1271/2522] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:37:44 2022] PUT /identity/v3/projects/b0f4625558bf47098a59298065f9765b/users/51ca5c1e25d045fa9f132841533616be/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:37:44.295928 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821fa14-e849-443f-a326-3ddf5fee9291 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:44.296249 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821fa14-e849-443f-a326-3ddf5fee9291 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:44.296656 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821fa14-e849-443f-a326-3ddf5fee9291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:44.324873 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-5821fa14-e849-443f-a326-3ddf5fee9291 None None] MFA Rules not processed for user `51ca5c1e25d045fa9f132841533616be`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:37:44.326078 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-5821fa14-e849-443f-a326-3ddf5fee9291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:44.387698 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1252/2523] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:37:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:37:44.392027 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faef38e3-cefd-4e10-8f8b-5e185af911ff admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:44.450423 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:37:44.455436 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faef38e3-cefd-4e10-8f8b-5e185af911ff admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:44.460058 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd45e79-af90-46f3-990f-f36c20a24303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:44.462439 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd45e79-af90-46f3-990f-f36c20a24303 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:44.462439 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd45e79-af90-46f3-990f-f36c20a24303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:44.462439 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd45e79-af90-46f3-990f-f36c20a24303 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:44.462439 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd45e79-af90-46f3-990f-f36c20a24303 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:44.463895 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd45e79-af90-46f3-990f-f36c20a24303 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:44.560338 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1272/2524] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:37:44 2022] POST /identity/v3/projects => generated 385 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:37:44.561576 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5821fa14-e849-443f-a326-3ddf5fee9291 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:44.572606 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5821fa14-e849-443f-a326-3ddf5fee9291 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:44.599893 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:37:44.631994 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1a7dd4-2550-4151-8e2a-8beb0d998438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:44.632771 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a7dd4-2550-4151-8e2a-8beb0d998438 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:44.633158 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a7dd4-2550-4151-8e2a-8beb0d998438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:44.633439 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a7dd4-2550-4151-8e2a-8beb0d998438 admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:44.634177 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1a7dd4-2550-4151-8e2a-8beb0d998438 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:44.636221 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1a7dd4-2550-4151-8e2a-8beb0d998438 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:44.733196 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1253/2525] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:37:44 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:37:44.737174 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd45e79-af90-46f3-990f-f36c20a24303 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:44.747653 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd45e79-af90-46f3-990f-f36c20a24303 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:44.752381 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f661c57-c66d-4ee6-badf-bc4b663c7489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:44.752996 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f661c57-c66d-4ee6-badf-bc4b663c7489 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:44.753282 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f661c57-c66d-4ee6-badf-bc4b663c7489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:44.753558 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f661c57-c66d-4ee6-badf-bc4b663c7489 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:44.754294 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f661c57-c66d-4ee6-badf-bc4b663c7489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:44.755980 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f661c57-c66d-4ee6-badf-bc4b663c7489 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:44.766618 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1273/2526] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:37:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:37:44.770337 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1a7dd4-2550-4151-8e2a-8beb0d998438 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:44.781270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1a7dd4-2550-4151-8e2a-8beb0d998438 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:44.785990 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5439ce49-d836-48ae-8a18-10117f5319b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:44.786596 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439ce49-d836-48ae-8a18-10117f5319b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:44.786883 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439ce49-d836-48ae-8a18-10117f5319b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:44.787161 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439ce49-d836-48ae-8a18-10117f5319b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:44.787854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5439ce49-d836-48ae-8a18-10117f5319b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:44.789486 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5439ce49-d836-48ae-8a18-10117f5319b0 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:44.802469 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1254/2527] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:37:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:37:44.803632 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f661c57-c66d-4ee6-badf-bc4b663c7489 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:44.813874 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f661c57-c66d-4ee6-badf-bc4b663c7489 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:44.822368 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7773cb-c8f0-4ef1-ac13-922963c63e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:44.823077 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7773cb-c8f0-4ef1-ac13-922963c63e7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:44.823365 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7773cb-c8f0-4ef1-ac13-922963c63e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:44.823650 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7773cb-c8f0-4ef1-ac13-922963c63e7f admin admin] PATH_INFO: `/v3/projects/cca021dedf7e4d3f8614c2c17aa9b3b3/users/779d2aed06924c7e82d57e46174a5b72/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:44.840399 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7773cb-c8f0-4ef1-ac13-922963c63e7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cca021dedf7e4d3f8614c2c17aa9b3b3, user_id=779d2aed06924c7e82d57e46174a5b72, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:44.843055 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7773cb-c8f0-4ef1-ac13-922963c63e7f admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:44.922229 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1274/2528] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:37:44 2022] PUT /identity/v3/projects/cca021dedf7e4d3f8614c2c17aa9b3b3/users/779d2aed06924c7e82d57e46174a5b72/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:37:44.932637 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc0e2b-acca-4764-9493-fdc8eff087dd None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:44.932932 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc0e2b-acca-4764-9493-fdc8eff087dd None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:44.933902 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc0e2b-acca-4764-9493-fdc8eff087dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:44.963202 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-90fc0e2b-acca-4764-9493-fdc8eff087dd None None] MFA Rules not processed for user `779d2aed06924c7e82d57e46174a5b72`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:37:44.964482 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-90fc0e2b-acca-4764-9493-fdc8eff087dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:45.033117 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1255/2529] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:37:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:37:45.037895 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7773cb-c8f0-4ef1-ac13-922963c63e7f admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:45.105830 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7773cb-c8f0-4ef1-ac13-922963c63e7f admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:45.110600 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568b58b-94cb-4daa-9404-3e5ebd1f2536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:45.111220 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568b58b-94cb-4daa-9404-3e5ebd1f2536 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:45.111509 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568b58b-94cb-4daa-9404-3e5ebd1f2536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:45.111785 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568b58b-94cb-4daa-9404-3e5ebd1f2536 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:45.112518 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568b58b-94cb-4daa-9404-3e5ebd1f2536 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:45.114461 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568b58b-94cb-4daa-9404-3e5ebd1f2536 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:45.191804 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc0e2b-acca-4764-9493-fdc8eff087dd None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:45.193923 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-90fc0e2b-acca-4764-9493-fdc8eff087dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:45.201841 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1275/2530] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:37:45 2022] POST /identity/v3/projects => generated 385 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:37:45.204392 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568b58b-94cb-4daa-9404-3e5ebd1f2536 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:45.210483 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc0e2b-acca-4764-9493-fdc8eff087dd None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:45.276475 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568b58b-94cb-4daa-9404-3e5ebd1f2536 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:45.281605 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a80f2-c821-41b9-a762-f4e04580dd2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:45.282252 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a80f2-c821-41b9-a762-f4e04580dd2e admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:45.282543 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a80f2-c821-41b9-a762-f4e04580dd2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:45.282828 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a80f2-c821-41b9-a762-f4e04580dd2e admin admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:45.283520 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a80f2-c821-41b9-a762-f4e04580dd2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:45.285719 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a80f2-c821-41b9-a762-f4e04580dd2e admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:45.354528 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1276/2531] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:37:45 2022] POST /identity/v3/users => generated 299 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:37:45.357060 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a80f2-c821-41b9-a762-f4e04580dd2e admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:45.367430 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a80f2-c821-41b9-a762-f4e04580dd2e admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:45.372994 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe63a7-6381-4795-8869-64a333121e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:45.373613 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe63a7-6381-4795-8869-64a333121e0f admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:45.373909 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe63a7-6381-4795-8869-64a333121e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:45.374192 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe63a7-6381-4795-8869-64a333121e0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:45.374893 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebe63a7-6381-4795-8869-64a333121e0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:45.376591 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebe63a7-6381-4795-8869-64a333121e0f admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:45.387899 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1277/2532] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:37:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:37:45.391743 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe63a7-6381-4795-8869-64a333121e0f admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:45.399172 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0009a7-73b7-4873-a9f6-0d4bbcbc5464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:45.399928 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0009a7-73b7-4873-a9f6-0d4bbcbc5464 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:45.400174 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0009a7-73b7-4873-a9f6-0d4bbcbc5464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:45.400414 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0009a7-73b7-4873-a9f6-0d4bbcbc5464 admin admin] PATH_INFO: `/v3/users/ff040e1f99bf46d48446e2e9c0deb9cc` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:45.401630 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0009a7-73b7-4873-a9f6-0d4bbcbc5464 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff040e1f99bf46d48446e2e9c0deb9cc)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:45.401756 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe63a7-6381-4795-8869-64a333121e0f admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:45.403676 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0009a7-73b7-4873-a9f6-0d4bbcbc5464 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:45.406457 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ece2c7-2a37-4f92-b0c0-06557f49e3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:45.410364 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ece2c7-2a37-4f92-b0c0-06557f49e3d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:45.410364 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ece2c7-2a37-4f92-b0c0-06557f49e3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:45.410364 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ece2c7-2a37-4f92-b0c0-06557f49e3d4 admin admin] PATH_INFO: `/v3/projects/2eed5ac0fef94615aff58a8a2071d04e/users/5753216959bd4d98bcfedff67a302506/roles/7c88c815db854954b516ab97cd471647` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:45.411601 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:37:45.425227 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ece2c7-2a37-4f92-b0c0-06557f49e3d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2eed5ac0fef94615aff58a8a2071d04e, user_id=5753216959bd4d98bcfedff67a302506, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:45.429657 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ece2c7-2a37-4f92-b0c0-06557f49e3d4 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:45.516319 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1278/2533] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:37:45 2022] PUT /identity/v3/projects/2eed5ac0fef94615aff58a8a2071d04e/users/5753216959bd4d98bcfedff67a302506/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:37:45.520227 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ece2c7-2a37-4f92-b0c0-06557f49e3d4 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:45.572651 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:37:45.581463 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-8b0009a7-73b7-4873-a9f6-0d4bbcbc5464 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff040e1f99bf46d48446e2e9c0deb9cc'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:45.582192 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ece2c7-2a37-4f92-b0c0-06557f49e3d4 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:45.586923 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b1ffd2-e6b1-4fdf-935e-b832916cddf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:45.587539 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1ffd2-e6b1-4fdf-935e-b832916cddf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:45.587832 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1ffd2-e6b1-4fdf-935e-b832916cddf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:45.588144 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1ffd2-e6b1-4fdf-935e-b832916cddf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:45.588847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b1ffd2-e6b1-4fdf-935e-b832916cddf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:45.589267 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-8b0009a7-73b7-4873-a9f6-0d4bbcbc5464 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff040e1f99bf46d48446e2e9c0deb9cc'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:45.589946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-8b0009a7-73b7-4873-a9f6-0d4bbcbc5464 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff040e1f99bf46d48446e2e9c0deb9cc'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:45.590506 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b1ffd2-e6b1-4fdf-935e-b832916cddf1 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:45.604494 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1279/2534] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:37:45 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:37:45.604664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-8b0009a7-73b7-4873-a9f6-0d4bbcbc5464 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff040e1f99bf46d48446e2e9c0deb9cc'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:45.604664 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-8b0009a7-73b7-4873-a9f6-0d4bbcbc5464 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff040e1f99bf46d48446e2e9c0deb9cc'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:45.605900 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b1ffd2-e6b1-4fdf-935e-b832916cddf1 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:45.607583 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-76b1ffd2-e6b1-4fdf-935e-b832916cddf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:45.620423 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b1ffd2-e6b1-4fdf-935e-b832916cddf1 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:45.626763 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968d81e-4140-4e03-85cc-366bee12dfd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:45.627441 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968d81e-4140-4e03-85cc-366bee12dfd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:45.627734 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968d81e-4140-4e03-85cc-366bee12dfd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:45.628018 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968d81e-4140-4e03-85cc-366bee12dfd8 admin admin] PATH_INFO: `/v3/projects/2eed5ac0fef94615aff58a8a2071d04e/users/5753216959bd4d98bcfedff67a302506/roles/7c88c815db854954b516ab97cd471647` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:45.630149 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a968d81e-4140-4e03-85cc-366bee12dfd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2eed5ac0fef94615aff58a8a2071d04e, user_id=5753216959bd4d98bcfedff67a302506, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:45.632977 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a968d81e-4140-4e03-85cc-366bee12dfd8 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:45.643507 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1280/2535] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:37:45 2022] PUT /identity/v3/projects/2eed5ac0fef94615aff58a8a2071d04e/users/5753216959bd4d98bcfedff67a302506/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:37:45.654117 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64125ab3-c2ce-49a0-8e13-207106cb38d0 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:45.654410 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64125ab3-c2ce-49a0-8e13-207106cb38d0 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:45.654692 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64125ab3-c2ce-49a0-8e13-207106cb38d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:45.681623 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1256/2536] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:37:45 2022] DELETE /identity/v3/users/ff040e1f99bf46d48446e2e9c0deb9cc => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:37:45.682366 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-64125ab3-c2ce-49a0-8e13-207106cb38d0 None None] MFA Rules not processed for user `5753216959bd4d98bcfedff67a302506`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:37:45.684201 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-64125ab3-c2ce-49a0-8e13-207106cb38d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:45.760390 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1281/2537] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:37:45 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:37:45.771218 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0009a7-73b7-4873-a9f6-0d4bbcbc5464 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:45.773190 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-8b0009a7-73b7-4873-a9f6-0d4bbcbc5464 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:45.831714 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0009a7-73b7-4873-a9f6-0d4bbcbc5464 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:45.835441 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb45248-f045-4679-8751-20d7ed0c6ae9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:45.836078 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb45248-f045-4679-8751-20d7ed0c6ae9 service cinder] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:45.836390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb45248-f045-4679-8751-20d7ed0c6ae9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:45.836685 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb45248-f045-4679-8751-20d7ed0c6ae9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:45.846401 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb45248-f045-4679-8751-20d7ed0c6ae9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:45.847839 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb45248-f045-4679-8751-20d7ed0c6ae9 service cinder] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:45.858222 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1257/2538] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:37:45 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:37:45.918467 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64125ab3-c2ce-49a0-8e13-207106cb38d0 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:45.920321 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-64125ab3-c2ce-49a0-8e13-207106cb38d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:45.987318 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:37:46.000243 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64125ab3-c2ce-49a0-8e13-207106cb38d0 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:46.004891 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce265e87-e218-4e56-9924-7edc67976b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:46.005577 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce265e87-e218-4e56-9924-7edc67976b70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:46.005858 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce265e87-e218-4e56-9924-7edc67976b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:46.006143 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce265e87-e218-4e56-9924-7edc67976b70 admin admin] PATH_INFO: `/v3/projects/07cd370ebabf4f048e72f7b1fbadcb08` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:46.007323 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce265e87-e218-4e56-9924-7edc67976b70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07cd370ebabf4f048e72f7b1fbadcb08)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:46.009389 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce265e87-e218-4e56-9924-7edc67976b70 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:46.094999 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-ce265e87-e218-4e56-9924-7edc67976b70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07cd370ebabf4f048e72f7b1fbadcb08', 'request_id': 'req-ce265e87-e218-4e56-9924-7edc67976b70'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:46.101478 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-ce265e87-e218-4e56-9924-7edc67976b70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07cd370ebabf4f048e72f7b1fbadcb08', 'request_id': 'req-ce265e87-e218-4e56-9924-7edc67976b70'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:46.109947 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-ce265e87-e218-4e56-9924-7edc67976b70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07cd370ebabf4f048e72f7b1fbadcb08', 'request_id': 'req-ce265e87-e218-4e56-9924-7edc67976b70'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:46.194820 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-ce265e87-e218-4e56-9924-7edc67976b70 admin admin] The token cache is being invalidate because project 07cd370ebabf4f048e72f7b1fbadcb08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:37:46.195139 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-ce265e87-e218-4e56-9924-7edc67976b70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:46.195795 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-ce265e87-e218-4e56-9924-7edc67976b70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:46.197331 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1282/2539] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:37:45 2022] DELETE /identity/v3/projects/07cd370ebabf4f048e72f7b1fbadcb08 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:37:46.202685 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ea050-22b6-4137-b08a-c8b8713e01b0 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:46.202978 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ea050-22b6-4137-b08a-c8b8713e01b0 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:46.203267 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ea050-22b6-4137-b08a-c8b8713e01b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:46.223186 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-498ea050-22b6-4137-b08a-c8b8713e01b0 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:37:46.224460 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-498ea050-22b6-4137-b08a-c8b8713e01b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:46.284205 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1258/2540] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:37:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:37:46.291772 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0408ae8-4e3b-40d0-b841-3b6c43c2975b None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:46.292080 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0408ae8-4e3b-40d0-b841-3b6c43c2975b None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:46.292378 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0408ae8-4e3b-40d0-b841-3b6c43c2975b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:46.313681 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-b0408ae8-4e3b-40d0-b841-3b6c43c2975b None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:37:46.314929 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-b0408ae8-4e3b-40d0-b841-3b6c43c2975b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:46.322974 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1283/2541] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:37:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:37:46.327925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498ea050-22b6-4137-b08a-c8b8713e01b0 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:46.338595 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498ea050-22b6-4137-b08a-c8b8713e01b0 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:46.342885 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b52ff-e9e2-425b-86f6-b675091e2f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:46.347814 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b52ff-e9e2-425b-86f6-b675091e2f32 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:46.347814 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b52ff-e9e2-425b-86f6-b675091e2f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:46.347814 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b52ff-e9e2-425b-86f6-b675091e2f32 admin admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:46.347814 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499b52ff-e9e2-425b-86f6-b675091e2f32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:37:46.347814 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499b52ff-e9e2-425b-86f6-b675091e2f32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:46.347814 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499b52ff-e9e2-425b-86f6-b675091e2f32 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:46.358379 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1259/2542] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:37:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:37:46.363722 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0408ae8-4e3b-40d0-b841-3b6c43c2975b None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:46.374246 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0408ae8-4e3b-40d0-b841-3b6c43c2975b None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:46.379269 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd465dc-5fd3-4870-af45-1ef75a0a6f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:46.379924 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd465dc-5fd3-4870-af45-1ef75a0a6f1e admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:46.380267 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd465dc-5fd3-4870-af45-1ef75a0a6f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:46.380544 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd465dc-5fd3-4870-af45-1ef75a0a6f1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:46.381233 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd465dc-5fd3-4870-af45-1ef75a0a6f1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:46.383230 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd465dc-5fd3-4870-af45-1ef75a0a6f1e admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:46.492006 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1284/2543] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:37:46 2022] POST /identity/v3/projects => generated 369 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:37:46.493323 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b52ff-e9e2-425b-86f6-b675091e2f32 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:46.503749 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b52ff-e9e2-425b-86f6-b675091e2f32 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:46.545168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:37:46.560333 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3077653c-c8cd-4d25-91e9-3634ff7baa89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:46.560975 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077653c-c8cd-4d25-91e9-3634ff7baa89 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:46.561232 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077653c-c8cd-4d25-91e9-3634ff7baa89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:46.563619 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077653c-c8cd-4d25-91e9-3634ff7baa89 admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:46.563619 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3077653c-c8cd-4d25-91e9-3634ff7baa89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:46.564228 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3077653c-c8cd-4d25-91e9-3634ff7baa89 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:46.642445 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1260/2544] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:37:46 2022] POST /identity/v3/users => generated 285 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:37:46.646844 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd465dc-5fd3-4870-af45-1ef75a0a6f1e admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:46.657668 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd465dc-5fd3-4870-af45-1ef75a0a6f1e admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:46.662344 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edffd38-a3eb-465d-a980-6314f1a5e378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:46.662979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edffd38-a3eb-465d-a980-6314f1a5e378 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:46.663258 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edffd38-a3eb-465d-a980-6314f1a5e378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:46.663533 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edffd38-a3eb-465d-a980-6314f1a5e378 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:46.664272 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edffd38-a3eb-465d-a980-6314f1a5e378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:46.666947 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edffd38-a3eb-465d-a980-6314f1a5e378 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:46.679323 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1285/2545] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:37:46.684195 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3077653c-c8cd-4d25-91e9-3634ff7baa89 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:46.695856 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3077653c-c8cd-4d25-91e9-3634ff7baa89 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:46.701321 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168463a0-d2d1-4096-a1db-68ce2fc7908c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:46.701938 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168463a0-d2d1-4096-a1db-68ce2fc7908c admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:46.702224 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168463a0-d2d1-4096-a1db-68ce2fc7908c admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:46.702508 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168463a0-d2d1-4096-a1db-68ce2fc7908c admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:46.703235 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168463a0-d2d1-4096-a1db-68ce2fc7908c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:46.704878 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168463a0-d2d1-4096-a1db-68ce2fc7908c admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:46.715024 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1261/2546] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:37:46.719564 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edffd38-a3eb-465d-a980-6314f1a5e378 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:46.729636 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edffd38-a3eb-465d-a980-6314f1a5e378 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:46.734793 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70f30cb-c425-413b-967b-d4f041cb7300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:46.735460 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f30cb-c425-413b-967b-d4f041cb7300 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:46.735749 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f30cb-c425-413b-967b-d4f041cb7300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:46.736033 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f30cb-c425-413b-967b-d4f041cb7300 admin admin] PATH_INFO: `/v3/projects/0b8c2f1cc2f4495da1f2312f76dc3614/users/6b440e7d2c4d443dbb35a1378131f55d/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:46.756887 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70f30cb-c425-413b-967b-d4f041cb7300 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b8c2f1cc2f4495da1f2312f76dc3614, user_id=6b440e7d2c4d443dbb35a1378131f55d, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:46.759568 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70f30cb-c425-413b-967b-d4f041cb7300 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:46.835428 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1286/2547] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:37:46 2022] PUT /identity/v3/projects/0b8c2f1cc2f4495da1f2312f76dc3614/users/6b440e7d2c4d443dbb35a1378131f55d/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:37:46.844300 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfb4ae-c33c-4117-8162-6cf0dc15cec9 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:46.844816 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfb4ae-c33c-4117-8162-6cf0dc15cec9 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:46.844924 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfb4ae-c33c-4117-8162-6cf0dc15cec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:46.873142 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-24dfb4ae-c33c-4117-8162-6cf0dc15cec9 None None] MFA Rules not processed for user `6b440e7d2c4d443dbb35a1378131f55d`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:37:46.874359 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-24dfb4ae-c33c-4117-8162-6cf0dc15cec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:46.889294 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70f30cb-c425-413b-967b-d4f041cb7300 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:46.890973 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-a70f30cb-c425-413b-967b-d4f041cb7300 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:46.939129 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1262/2548] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:37:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:37:46.949101 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70f30cb-c425-413b-967b-d4f041cb7300 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:46.950508 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dfb4ae-c33c-4117-8162-6cf0dc15cec9 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:46.951938 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-24dfb4ae-c33c-4117-8162-6cf0dc15cec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:46.953338 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365aa64f-4bf7-4764-9a5b-78b64f8923ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:46.953965 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365aa64f-4bf7-4764-9a5b-78b64f8923ba service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:46.954248 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365aa64f-4bf7-4764-9a5b-78b64f8923ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:46.954527 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365aa64f-4bf7-4764-9a5b-78b64f8923ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:46.956245 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-365aa64f-4bf7-4764-9a5b-78b64f8923ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:46.990590 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:37:47.009475 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dfb4ae-c33c-4117-8162-6cf0dc15cec9 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:47.013124 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3e62c5-d6a4-4322-81a2-89047296c029 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c386fa0483cf42efaddbc2395f6d1fce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:47.013740 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e62c5-d6a4-4322-81a2-89047296c029 service glance] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:47.014028 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e62c5-d6a4-4322-81a2-89047296c029 service glance] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:47.014404 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e62c5-d6a4-4322-81a2-89047296c029 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:47.016236 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365aa64f-4bf7-4764-9a5b-78b64f8923ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:47.023890 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365aa64f-4bf7-4764-9a5b-78b64f8923ba service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:47.024083 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3e62c5-d6a4-4322-81a2-89047296c029 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:47.024778 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3e62c5-d6a4-4322-81a2-89047296c029 service glance] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:47.026920 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1287/2549] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:37:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:37:47.033626 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1263/2550] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:37:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 15:37:51.925669 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365aa64f-4bf7-4764-9a5b-78b64f8923ba service cinder] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:51.934427 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365aa64f-4bf7-4764-9a5b-78b64f8923ba service cinder] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:51.942820 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8397f1f-701c-4614-a1b8-70e8e776ac77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:51.942820 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8397f1f-701c-4614-a1b8-70e8e776ac77 service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:51.942820 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8397f1f-701c-4614-a1b8-70e8e776ac77 service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:51.942820 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8397f1f-701c-4614-a1b8-70e8e776ac77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:51.943159 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-a8397f1f-701c-4614-a1b8-70e8e776ac77 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:52.033788 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8397f1f-701c-4614-a1b8-70e8e776ac77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:52.035419 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8397f1f-701c-4614-a1b8-70e8e776ac77 service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:52.052306 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1288/2551] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:37:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:37:55.175462 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3e62c5-d6a4-4322-81a2-89047296c029 service glance] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:55.177557 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-da3e62c5-d6a4-4322-81a2-89047296c029 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:55.232840 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3e62c5-d6a4-4322-81a2-89047296c029 service glance] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:55.237806 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f64b20f-9ad9-46d1-a153-11b82355b0f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:55.238399 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64b20f-9ad9-46d1-a153-11b82355b0f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:55.238678 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64b20f-9ad9-46d1-a153-11b82355b0f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:55.238950 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64b20f-9ad9-46d1-a153-11b82355b0f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:55.240721 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-8f64b20f-9ad9-46d1-a153-11b82355b0f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:55.257719 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:37:55.315777 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f64b20f-9ad9-46d1-a153-11b82355b0f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:55.317251 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f64b20f-9ad9-46d1-a153-11b82355b0f7 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:55.328390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1264/2552] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:37:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:37:56.104059 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8397f1f-701c-4614-a1b8-70e8e776ac77 service cinder] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:56.105876 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-a8397f1f-701c-4614-a1b8-70e8e776ac77 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:56.121328 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8397f1f-701c-4614-a1b8-70e8e776ac77 service cinder] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:56.127951 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c638e4-a573-41cc-9778-5b7626be2f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:56.128703 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c638e4-a573-41cc-9778-5b7626be2f8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:56.128995 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c638e4-a573-41cc-9778-5b7626be2f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:56.129277 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c638e4-a573-41cc-9778-5b7626be2f8f admin admin] PATH_INFO: `/v3/users/51ca5c1e25d045fa9f132841533616be` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:56.130475 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c638e4-a573-41cc-9778-5b7626be2f8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=51ca5c1e25d045fa9f132841533616be)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:56.132520 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c638e4-a573-41cc-9778-5b7626be2f8f admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:56.274565 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-46c638e4-a573-41cc-9778-5b7626be2f8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51ca5c1e25d045fa9f132841533616be'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:56.275035 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-46c638e4-a573-41cc-9778-5b7626be2f8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51ca5c1e25d045fa9f132841533616be'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:56.290182 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-46c638e4-a573-41cc-9778-5b7626be2f8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51ca5c1e25d045fa9f132841533616be'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:56.373947 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-46c638e4-a573-41cc-9778-5b7626be2f8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51ca5c1e25d045fa9f132841533616be'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:56.383810 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-46c638e4-a573-41cc-9778-5b7626be2f8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51ca5c1e25d045fa9f132841533616be'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:56.397416 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1289/2553] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:37:56 2022] DELETE /identity/v3/users/51ca5c1e25d045fa9f132841533616be => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:37:56.399331 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c828090-b747-4392-80de-f795a141c699 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:56.400238 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c828090-b747-4392-80de-f795a141c699 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:56.400700 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c828090-b747-4392-80de-f795a141c699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:56.422321 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-9c828090-b747-4392-80de-f795a141c699 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:37:56.424443 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-9c828090-b747-4392-80de-f795a141c699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:56.489953 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1265/2554] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:37:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:37:56.499833 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c638e4-a573-41cc-9778-5b7626be2f8f admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:56.501449 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-46c638e4-a573-41cc-9778-5b7626be2f8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:56.559217 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:37:56.569460 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c638e4-a573-41cc-9778-5b7626be2f8f admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:56.573134 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e3e5a1-974d-465f-8592-e5e583be9306 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:56.573782 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e3e5a1-974d-465f-8592-e5e583be9306 service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:56.574072 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e3e5a1-974d-465f-8592-e5e583be9306 service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:56.574340 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e3e5a1-974d-465f-8592-e5e583be9306 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:56.584519 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e3e5a1-974d-465f-8592-e5e583be9306 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:56.589377 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e3e5a1-974d-465f-8592-e5e583be9306 service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:56.596195 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1290/2555] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:37:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:37:57.051040 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c828090-b747-4392-80de-f795a141c699 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:57.052527 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-9c828090-b747-4392-80de-f795a141c699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:57.065270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c828090-b747-4392-80de-f795a141c699 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:57.070890 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a10a7-9285-4cc8-ae30-e46b50aef902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:57.076049 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a10a7-9285-4cc8-ae30-e46b50aef902 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:57.076049 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a10a7-9285-4cc8-ae30-e46b50aef902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:57.076049 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a10a7-9285-4cc8-ae30-e46b50aef902 admin admin] PATH_INFO: `/v3/projects/b0f4625558bf47098a59298065f9765b` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:57.076049 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a10a7-9285-4cc8-ae30-e46b50aef902 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0f4625558bf47098a59298065f9765b)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:57.076049 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a10a7-9285-4cc8-ae30-e46b50aef902 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:57.217749 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-515a10a7-9285-4cc8-ae30-e46b50aef902 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0f4625558bf47098a59298065f9765b', 'request_id': 'req-515a10a7-9285-4cc8-ae30-e46b50aef902'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:57.229138 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-515a10a7-9285-4cc8-ae30-e46b50aef902 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0f4625558bf47098a59298065f9765b', 'request_id': 'req-515a10a7-9285-4cc8-ae30-e46b50aef902'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:57.288944 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-515a10a7-9285-4cc8-ae30-e46b50aef902 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0f4625558bf47098a59298065f9765b', 'request_id': 'req-515a10a7-9285-4cc8-ae30-e46b50aef902'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:57.305028 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-515a10a7-9285-4cc8-ae30-e46b50aef902 admin admin] The token cache is being invalidate because project b0f4625558bf47098a59298065f9765b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:37:57.305321 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-515a10a7-9285-4cc8-ae30-e46b50aef902 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:57.305987 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-515a10a7-9285-4cc8-ae30-e46b50aef902 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:57.307488 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1266/2556] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:37:57 2022] DELETE /identity/v3/projects/b0f4625558bf47098a59298065f9765b => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:37:57.314930 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e3e5a1-974d-465f-8592-e5e583be9306 service neutron] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:57.332024 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e3e5a1-974d-465f-8592-e5e583be9306 service neutron] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:57.405490 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6297f353-ae06-47fc-b7f0-e2a6db406521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:57.406464 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297f353-ae06-47fc-b7f0-e2a6db406521 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:57.407185 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297f353-ae06-47fc-b7f0-e2a6db406521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:57.407681 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297f353-ae06-47fc-b7f0-e2a6db406521 admin admin] PATH_INFO: `/v3/users/779d2aed06924c7e82d57e46174a5b72` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:57.409362 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6297f353-ae06-47fc-b7f0-e2a6db406521 admin admin] RBAC: Authorizing `identity:delete_user(user_id=779d2aed06924c7e82d57e46174a5b72)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:57.411717 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6297f353-ae06-47fc-b7f0-e2a6db406521 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:57.597322 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-6297f353-ae06-47fc-b7f0-e2a6db406521 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '779d2aed06924c7e82d57e46174a5b72'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:57.598096 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-6297f353-ae06-47fc-b7f0-e2a6db406521 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '779d2aed06924c7e82d57e46174a5b72'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:57.623545 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-6297f353-ae06-47fc-b7f0-e2a6db406521 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '779d2aed06924c7e82d57e46174a5b72'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:57.682223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-6297f353-ae06-47fc-b7f0-e2a6db406521 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '779d2aed06924c7e82d57e46174a5b72'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:57.692008 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-6297f353-ae06-47fc-b7f0-e2a6db406521 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '779d2aed06924c7e82d57e46174a5b72'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:57.705409 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1291/2557] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:37:57 2022] DELETE /identity/v3/users/779d2aed06924c7e82d57e46174a5b72 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:37:58.162273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a10a7-9285-4cc8-ae30-e46b50aef902 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:58.163758 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-515a10a7-9285-4cc8-ae30-e46b50aef902 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:58.180682 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a10a7-9285-4cc8-ae30-e46b50aef902 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:58.216929 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:37:58.252903 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eee942e-a621-4598-b295-0cef19c8d50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:58.253635 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee942e-a621-4598-b295-0cef19c8d50a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:58.253969 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee942e-a621-4598-b295-0cef19c8d50a admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:58.254274 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee942e-a621-4598-b295-0cef19c8d50a admin admin] PATH_INFO: `/v3/projects/cca021dedf7e4d3f8614c2c17aa9b3b3` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:58.255606 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eee942e-a621-4598-b295-0cef19c8d50a admin admin] RBAC: Authorizing `identity:delete_project(project_id=cca021dedf7e4d3f8614c2c17aa9b3b3)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:58.258008 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eee942e-a621-4598-b295-0cef19c8d50a admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:58.364345 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-3eee942e-a621-4598-b295-0cef19c8d50a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cca021dedf7e4d3f8614c2c17aa9b3b3', 'request_id': 'req-3eee942e-a621-4598-b295-0cef19c8d50a'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:58.370732 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-3eee942e-a621-4598-b295-0cef19c8d50a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cca021dedf7e4d3f8614c2c17aa9b3b3', 'request_id': 'req-3eee942e-a621-4598-b295-0cef19c8d50a'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:58.442264 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-3eee942e-a621-4598-b295-0cef19c8d50a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cca021dedf7e4d3f8614c2c17aa9b3b3', 'request_id': 'req-3eee942e-a621-4598-b295-0cef19c8d50a'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:58.453402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-3eee942e-a621-4598-b295-0cef19c8d50a admin admin] The token cache is being invalidate because project cca021dedf7e4d3f8614c2c17aa9b3b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:37:58.453691 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-3eee942e-a621-4598-b295-0cef19c8d50a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:58.454345 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-3eee942e-a621-4598-b295-0cef19c8d50a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:58.455877 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1267/2558] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:37:58 2022] DELETE /identity/v3/projects/cca021dedf7e4d3f8614c2c17aa9b3b3 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:37:58.459974 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6297f353-ae06-47fc-b7f0-e2a6db406521 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:58.461321 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-6297f353-ae06-47fc-b7f0-e2a6db406521 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:58.474948 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6297f353-ae06-47fc-b7f0-e2a6db406521 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:58.545607 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b45a97d-5502-4d43-a4dc-e782cf6bb34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:58.546482 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45a97d-5502-4d43-a4dc-e782cf6bb34c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:58.546831 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45a97d-5502-4d43-a4dc-e782cf6bb34c admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:58.547119 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45a97d-5502-4d43-a4dc-e782cf6bb34c admin admin] PATH_INFO: `/v3/users/5753216959bd4d98bcfedff67a302506` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:58.548493 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b45a97d-5502-4d43-a4dc-e782cf6bb34c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5753216959bd4d98bcfedff67a302506)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:58.550690 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b45a97d-5502-4d43-a4dc-e782cf6bb34c admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:58.558257 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:37:58.731716 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-7b45a97d-5502-4d43-a4dc-e782cf6bb34c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5753216959bd4d98bcfedff67a302506'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:58.732490 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-7b45a97d-5502-4d43-a4dc-e782cf6bb34c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5753216959bd4d98bcfedff67a302506'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:58.746662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-7b45a97d-5502-4d43-a4dc-e782cf6bb34c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5753216959bd4d98bcfedff67a302506'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:58.813539 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-7b45a97d-5502-4d43-a4dc-e782cf6bb34c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5753216959bd4d98bcfedff67a302506'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:58.822603 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-7b45a97d-5502-4d43-a4dc-e782cf6bb34c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5753216959bd4d98bcfedff67a302506'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:58.839546 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1292/2559] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:37:58 2022] DELETE /identity/v3/users/5753216959bd4d98bcfedff67a302506 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:37:59.140303 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eee942e-a621-4598-b295-0cef19c8d50a admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:59.141852 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-3eee942e-a621-4598-b295-0cef19c8d50a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:59.218107 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eee942e-a621-4598-b295-0cef19c8d50a admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:59.262155 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b45a97d-5502-4d43-a4dc-e782cf6bb34c admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:59.263910 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-7b45a97d-5502-4d43-a4dc-e782cf6bb34c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:59.278043 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:37:59.287212 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ef143-a933-49f5-aca8-c667661eaf17 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '107a7c18a2c6440b86e1b7c3e39cc724', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:59.287927 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ef143-a933-49f5-aca8-c667661eaf17 service placement] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:59.288342 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ef143-a933-49f5-aca8-c667661eaf17 service placement] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:59.288774 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ef143-a933-49f5-aca8-c667661eaf17 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:59.291976 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-475ef143-a933-49f5-aca8-c667661eaf17 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:59.306050 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475ef143-a933-49f5-aca8-c667661eaf17 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:59.309133 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475ef143-a933-49f5-aca8-c667661eaf17 service placement] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:59.320407 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1268/2560] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:37:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:37:59.321240 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ef143-a933-49f5-aca8-c667661eaf17 service placement] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:59.323431 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-475ef143-a933-49f5-aca8-c667661eaf17 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:59.365432 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b45a97d-5502-4d43-a4dc-e782cf6bb34c admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:59.370070 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890e621d-ad66-4f98-b298-365bb7078d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:59.370653 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e621d-ad66-4f98-b298-365bb7078d00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:59.370940 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e621d-ad66-4f98-b298-365bb7078d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:59.371217 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e621d-ad66-4f98-b298-365bb7078d00 admin admin] PATH_INFO: `/v3/users/d36b38a6e2d34c41b827ff6db9742332` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:59.372819 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890e621d-ad66-4f98-b298-365bb7078d00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d36b38a6e2d34c41b827ff6db9742332)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:59.375200 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890e621d-ad66-4f98-b298-365bb7078d00 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:59.403701 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ef143-a933-49f5-aca8-c667661eaf17 service placement] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:59.409353 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59021dca-e472-4492-b640-9aaa4fc2c6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:59.410064 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59021dca-e472-4492-b640-9aaa4fc2c6fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:59.410486 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59021dca-e472-4492-b640-9aaa4fc2c6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:59.410833 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59021dca-e472-4492-b640-9aaa4fc2c6fd admin admin] PATH_INFO: `/v3/projects/2eed5ac0fef94615aff58a8a2071d04e` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:59.412272 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59021dca-e472-4492-b640-9aaa4fc2c6fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=2eed5ac0fef94615aff58a8a2071d04e)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:59.417148 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59021dca-e472-4492-b640-9aaa4fc2c6fd admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:59.556500 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-890e621d-ad66-4f98-b298-365bb7078d00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd36b38a6e2d34c41b827ff6db9742332'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:59.557652 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-890e621d-ad66-4f98-b298-365bb7078d00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd36b38a6e2d34c41b827ff6db9742332'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:59.558241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-59021dca-e472-4492-b640-9aaa4fc2c6fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2eed5ac0fef94615aff58a8a2071d04e', 'request_id': 'req-59021dca-e472-4492-b640-9aaa4fc2c6fd'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:59.565769 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-59021dca-e472-4492-b640-9aaa4fc2c6fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2eed5ac0fef94615aff58a8a2071d04e', 'request_id': 'req-59021dca-e472-4492-b640-9aaa4fc2c6fd'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:59.572202 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-890e621d-ad66-4f98-b298-365bb7078d00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd36b38a6e2d34c41b827ff6db9742332'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:59.607423 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-59021dca-e472-4492-b640-9aaa4fc2c6fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2eed5ac0fef94615aff58a8a2071d04e', 'request_id': 'req-59021dca-e472-4492-b640-9aaa4fc2c6fd'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:59.620367 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-59021dca-e472-4492-b640-9aaa4fc2c6fd admin admin] The token cache is being invalidate because project 2eed5ac0fef94615aff58a8a2071d04e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:37:59.620608 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-59021dca-e472-4492-b640-9aaa4fc2c6fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:59.621281 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-59021dca-e472-4492-b640-9aaa4fc2c6fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:59.622812 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1269/2561] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:37:59 2022] DELETE /identity/v3/projects/2eed5ac0fef94615aff58a8a2071d04e => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:37:59.628899 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705213d-5acd-45f6-af58-208ecd2acc48 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:59.629220 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705213d-5acd-45f6-af58-208ecd2acc48 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:59.629501 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705213d-5acd-45f6-af58-208ecd2acc48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:59.648672 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-0705213d-5acd-45f6-af58-208ecd2acc48 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:37:59.649872 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-0705213d-5acd-45f6-af58-208ecd2acc48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:59.659147 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-890e621d-ad66-4f98-b298-365bb7078d00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd36b38a6e2d34c41b827ff6db9742332'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:59.666996 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-890e621d-ad66-4f98-b298-365bb7078d00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd36b38a6e2d34c41b827ff6db9742332'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:37:59.673970 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1293/2562] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:37:59 2022] DELETE /identity/v3/users/d36b38a6e2d34c41b827ff6db9742332 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:37:59.710415 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1270/2563] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:37:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:37:59.719979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be85e0-8548-4b61-80df-d5a85fc56d82 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:59.720318 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be85e0-8548-4b61-80df-d5a85fc56d82 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:59.720673 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be85e0-8548-4b61-80df-d5a85fc56d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:59.730657 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:37:59.742517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-92be85e0-8548-4b61-80df-d5a85fc56d82 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:37:59.744295 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-92be85e0-8548-4b61-80df-d5a85fc56d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:59.805642 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1294/2564] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:37:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:37:59.810118 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0705213d-5acd-45f6-af58-208ecd2acc48 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:59.821336 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0705213d-5acd-45f6-af58-208ecd2acc48 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:59.826453 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2e644d-0ba1-4759-be82-27f68d4d18fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:59.827125 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e644d-0ba1-4759-be82-27f68d4d18fb admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:59.827408 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e644d-0ba1-4759-be82-27f68d4d18fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:59.827715 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e644d-0ba1-4759-be82-27f68d4d18fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:59.828530 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2e644d-0ba1-4759-be82-27f68d4d18fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:37:59.828871 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2e644d-0ba1-4759-be82-27f68d4d18fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:59.830568 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2e644d-0ba1-4759-be82-27f68d4d18fb admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:59.842885 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1271/2565] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:37:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:37:59.847302 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be85e0-8548-4b61-80df-d5a85fc56d82 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:59.856804 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be85e0-8548-4b61-80df-d5a85fc56d82 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:59.862096 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a731f96c-32d5-4a6b-a1ac-e7832f7de5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:59.862707 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731f96c-32d5-4a6b-a1ac-e7832f7de5e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:59.862996 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731f96c-32d5-4a6b-a1ac-e7832f7de5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:59.863280 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731f96c-32d5-4a6b-a1ac-e7832f7de5e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:59.863996 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a731f96c-32d5-4a6b-a1ac-e7832f7de5e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:59.865943 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a731f96c-32d5-4a6b-a1ac-e7832f7de5e8 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:59.895476 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2e644d-0ba1-4759-be82-27f68d4d18fb admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:37:59.897618 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-3f2e644d-0ba1-4759-be82-27f68d4d18fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:37:59.918731 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2e644d-0ba1-4759-be82-27f68d4d18fb admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:37:59.927227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d251afc-af3e-4561-b07d-db42eb0ee459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:37:59.928003 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d251afc-af3e-4561-b07d-db42eb0ee459 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:37:59.928368 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d251afc-af3e-4561-b07d-db42eb0ee459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:37:59.928881 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d251afc-af3e-4561-b07d-db42eb0ee459 admin admin] PATH_INFO: `/v3/projects/648f7ac8fb934441b6782d00d0b5ef10` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:37:59.930136 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d251afc-af3e-4561-b07d-db42eb0ee459 admin admin] RBAC: Authorizing `identity:delete_project(project_id=648f7ac8fb934441b6782d00d0b5ef10)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:37:59.931488 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1295/2566] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:37:59 2022] POST /identity/v3/projects => generated 399 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:37:59.934766 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d251afc-af3e-4561-b07d-db42eb0ee459 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:37:59.935192 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:37:59.937030 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a731f96c-32d5-4a6b-a1ac-e7832f7de5e8 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:00.004764 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a731f96c-32d5-4a6b-a1ac-e7832f7de5e8 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:00.009742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2e265f-7027-4d22-9ca9-439330509716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:00.010361 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e265f-7027-4d22-9ca9-439330509716 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:00.010674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e265f-7027-4d22-9ca9-439330509716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:00.010940 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e265f-7027-4d22-9ca9-439330509716 admin admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:00.011940 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2e265f-7027-4d22-9ca9-439330509716 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:00.013981 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2e265f-7027-4d22-9ca9-439330509716 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:00.021039 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-7d251afc-af3e-4561-b07d-db42eb0ee459 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '648f7ac8fb934441b6782d00d0b5ef10', 'request_id': 'req-7d251afc-af3e-4561-b07d-db42eb0ee459'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:00.028129 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-7d251afc-af3e-4561-b07d-db42eb0ee459 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '648f7ac8fb934441b6782d00d0b5ef10', 'request_id': 'req-7d251afc-af3e-4561-b07d-db42eb0ee459'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:00.070529 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1296/2567] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:37:59 2022] POST /identity/v3/users => generated 300 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:38:00.074749 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2e265f-7027-4d22-9ca9-439330509716 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:00.084602 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2e265f-7027-4d22-9ca9-439330509716 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:00.093567 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-7d251afc-af3e-4561-b07d-db42eb0ee459 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '648f7ac8fb934441b6782d00d0b5ef10', 'request_id': 'req-7d251afc-af3e-4561-b07d-db42eb0ee459'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:00.104570 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-7d251afc-af3e-4561-b07d-db42eb0ee459 admin admin] The token cache is being invalidate because project 648f7ac8fb934441b6782d00d0b5ef10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:38:00.104863 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-7d251afc-af3e-4561-b07d-db42eb0ee459 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:00.105531 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-7d251afc-af3e-4561-b07d-db42eb0ee459 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:00.107064 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1272/2568] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:37:59 2022] DELETE /identity/v3/projects/648f7ac8fb934441b6782d00d0b5ef10 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:38:00.112970 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a2ec6-2063-4d4c-86ef-625e564fc018 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:00.113266 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a2ec6-2063-4d4c-86ef-625e564fc018 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:00.113548 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a2ec6-2063-4d4c-86ef-625e564fc018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:00.132364 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-142a2ec6-2063-4d4c-86ef-625e564fc018 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:38:00.133721 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-142a2ec6-2063-4d4c-86ef-625e564fc018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:00.141736 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175eda87-ae35-4784-a4a2-9d7134b146db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:00.142301 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175eda87-ae35-4784-a4a2-9d7134b146db admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:00.142591 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175eda87-ae35-4784-a4a2-9d7134b146db admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:00.142871 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175eda87-ae35-4784-a4a2-9d7134b146db admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:00.143571 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175eda87-ae35-4784-a4a2-9d7134b146db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:00.145230 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175eda87-ae35-4784-a4a2-9d7134b146db admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:00.156358 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1297/2569] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:38:00.160187 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175eda87-ae35-4784-a4a2-9d7134b146db admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:00.161832 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-175eda87-ae35-4784-a4a2-9d7134b146db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:00.175352 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175eda87-ae35-4784-a4a2-9d7134b146db admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:00.180185 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2994b93-5981-40f0-afac-2db4507eb4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:00.180814 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2994b93-5981-40f0-afac-2db4507eb4e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:00.181095 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2994b93-5981-40f0-afac-2db4507eb4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:00.181373 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2994b93-5981-40f0-afac-2db4507eb4e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:00.182066 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2994b93-5981-40f0-afac-2db4507eb4e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:00.183739 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2994b93-5981-40f0-afac-2db4507eb4e8 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:00.186083 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:38:00.196189 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1298/2570] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 15:38:00.200809 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1273/2571] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:38:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:38:00.203034 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2994b93-5981-40f0-afac-2db4507eb4e8 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:00.209889 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f356af-fab7-459e-a99b-4c2c29b2d35a None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:00.209889 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f356af-fab7-459e-a99b-4c2c29b2d35a None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:00.210111 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f356af-fab7-459e-a99b-4c2c29b2d35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:00.213308 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2994b93-5981-40f0-afac-2db4507eb4e8 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:00.218519 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8faef83-1568-45e2-b62d-143898c97bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:00.222862 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8faef83-1568-45e2-b62d-143898c97bf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:00.222862 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8faef83-1568-45e2-b62d-143898c97bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:00.222862 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8faef83-1568-45e2-b62d-143898c97bf5 admin admin] PATH_INFO: `/v3/projects/423c220e15704c81afdf783e5bf70699/users/d8c940c5c41d47c38ebd4dcb384ecf2c/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:00.230152 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-91f356af-fab7-459e-a99b-4c2c29b2d35a None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:38:00.231381 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-91f356af-fab7-459e-a99b-4c2c29b2d35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:00.236250 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8faef83-1568-45e2-b62d-143898c97bf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=423c220e15704c81afdf783e5bf70699, user_id=d8c940c5c41d47c38ebd4dcb384ecf2c, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:00.238907 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8faef83-1568-45e2-b62d-143898c97bf5 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:00.239977 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1274/2572] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:38:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:38:00.245183 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f356af-fab7-459e-a99b-4c2c29b2d35a None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:00.254327 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f356af-fab7-459e-a99b-4c2c29b2d35a None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:00.259001 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670bbe26-462e-4660-9a84-6d54c246d080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:00.259604 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bbe26-462e-4660-9a84-6d54c246d080 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:00.259890 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bbe26-462e-4660-9a84-6d54c246d080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:00.260210 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bbe26-462e-4660-9a84-6d54c246d080 admin admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:00.260904 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670bbe26-462e-4660-9a84-6d54c246d080 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:38:00.261212 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670bbe26-462e-4660-9a84-6d54c246d080 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:00.262825 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670bbe26-462e-4660-9a84-6d54c246d080 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:00.275282 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1275/2573] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:38:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 15:38:00.279368 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670bbe26-462e-4660-9a84-6d54c246d080 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:00.289783 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670bbe26-462e-4660-9a84-6d54c246d080 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:00.294420 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd6a695-c422-4028-9dc8-e210aa14725b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:00.294997 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd6a695-c422-4028-9dc8-e210aa14725b admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:00.295287 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd6a695-c422-4028-9dc8-e210aa14725b admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:00.295556 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd6a695-c422-4028-9dc8-e210aa14725b admin admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:00.296229 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd6a695-c422-4028-9dc8-e210aa14725b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:00.298085 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd6a695-c422-4028-9dc8-e210aa14725b admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:00.310857 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1299/2574] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:38:00 2022] PUT /identity/v3/projects/423c220e15704c81afdf783e5bf70699/users/d8c940c5c41d47c38ebd4dcb384ecf2c/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:38:00.322155 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cec77-8d92-4ec5-8ba6-72cbb1778a25 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:00.322456 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cec77-8d92-4ec5-8ba6-72cbb1778a25 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:00.322740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cec77-8d92-4ec5-8ba6-72cbb1778a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:00.351981 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-da8cec77-8d92-4ec5-8ba6-72cbb1778a25 None None] MFA Rules not processed for user `d8c940c5c41d47c38ebd4dcb384ecf2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:38:00.353267 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-da8cec77-8d92-4ec5-8ba6-72cbb1778a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:00.369044 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1276/2575] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:38:00 2022] POST /identity/v3/projects => generated 383 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:38:00.373279 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd6a695-c422-4028-9dc8-e210aa14725b admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:00.430576 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:38:00.445479 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd6a695-c422-4028-9dc8-e210aa14725b admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:00.452220 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ee7c8-f04b-42a8-9846-628685f628e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:00.452866 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ee7c8-f04b-42a8-9846-628685f628e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:00.453155 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ee7c8-f04b-42a8-9846-628685f628e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:00.453436 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ee7c8-f04b-42a8-9846-628685f628e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:00.454156 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85ee7c8-f04b-42a8-9846-628685f628e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:00.456163 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85ee7c8-f04b-42a8-9846-628685f628e7 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:00.540149 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1277/2576] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:38:00 2022] POST /identity/v3/users => generated 292 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:38:00.544612 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ee7c8-f04b-42a8-9846-628685f628e7 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:00.552149 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1300/2577] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:38:00 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:38:00.555893 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ee7c8-f04b-42a8-9846-628685f628e7 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:00.556676 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8cec77-8d92-4ec5-8ba6-72cbb1778a25 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:00.560735 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250a292-5050-40cc-8eb8-85f8ac513b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:00.561334 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250a292-5050-40cc-8eb8-85f8ac513b6e admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:00.562869 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250a292-5050-40cc-8eb8-85f8ac513b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:00.562869 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250a292-5050-40cc-8eb8-85f8ac513b6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:00.563387 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3250a292-5050-40cc-8eb8-85f8ac513b6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:00.565013 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3250a292-5050-40cc-8eb8-85f8ac513b6e admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:00.575839 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1278/2578] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:38:00.580294 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250a292-5050-40cc-8eb8-85f8ac513b6e admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:00.590276 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250a292-5050-40cc-8eb8-85f8ac513b6e admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:00.595826 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada900c3-dbfc-42f3-9146-6f79e684d73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:00.596459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada900c3-dbfc-42f3-9146-6f79e684d73b admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:00.596747 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada900c3-dbfc-42f3-9146-6f79e684d73b admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:00.597026 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada900c3-dbfc-42f3-9146-6f79e684d73b admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:00.597709 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada900c3-dbfc-42f3-9146-6f79e684d73b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:00.599264 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada900c3-dbfc-42f3-9146-6f79e684d73b admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:00.610296 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1279/2579] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 15:38:00.614211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada900c3-dbfc-42f3-9146-6f79e684d73b admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:00.624423 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada900c3-dbfc-42f3-9146-6f79e684d73b admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:00.624566 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8cec77-8d92-4ec5-8ba6-72cbb1778a25 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:00.629135 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d04796-64bd-42b5-b6df-db2ccad6695e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:00.629317 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f589ff8-c3c9-4b0c-ba6e-08be9553070a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:00.629784 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d04796-64bd-42b5-b6df-db2ccad6695e admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:00.629897 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f589ff8-c3c9-4b0c-ba6e-08be9553070a admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:00.630071 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f589ff8-c3c9-4b0c-ba6e-08be9553070a admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:00.630205 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d04796-64bd-42b5-b6df-db2ccad6695e admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:00.630352 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f589ff8-c3c9-4b0c-ba6e-08be9553070a admin admin] PATH_INFO: `/v3/projects/2c77da48edba4903a1c51fee2a856167/users/dd5c43d63a754eeaaba429f5c8153bba/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:00.630461 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d04796-64bd-42b5-b6df-db2ccad6695e admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:00.631102 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d04796-64bd-42b5-b6df-db2ccad6695e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:00.633101 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d04796-64bd-42b5-b6df-db2ccad6695e admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:00.645388 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f589ff8-c3c9-4b0c-ba6e-08be9553070a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c77da48edba4903a1c51fee2a856167, user_id=dd5c43d63a754eeaaba429f5c8153bba, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:00.648496 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f589ff8-c3c9-4b0c-ba6e-08be9553070a admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:00.711645 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1301/2580] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:38:00 2022] POST /identity/v3/projects => generated 401 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:38:00.715433 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d04796-64bd-42b5-b6df-db2ccad6695e admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:00.733876 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1280/2581] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:38:00 2022] PUT /identity/v3/projects/2c77da48edba4903a1c51fee2a856167/users/dd5c43d63a754eeaaba429f5c8153bba/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:38:00.738728 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e598528-4905-4234-ad62-3dc050e3cb28 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:00.739141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e598528-4905-4234-ad62-3dc050e3cb28 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:00.739432 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e598528-4905-4234-ad62-3dc050e3cb28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:00.758408 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-7e598528-4905-4234-ad62-3dc050e3cb28 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:38:00.759633 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-7e598528-4905-4234-ad62-3dc050e3cb28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:00.767972 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:38:00.781089 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d04796-64bd-42b5-b6df-db2ccad6695e admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:00.820900 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1281/2582] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:38:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:38:00.831498 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e598528-4905-4234-ad62-3dc050e3cb28 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:00.833051 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-7e598528-4905-4234-ad62-3dc050e3cb28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:00.842109 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd86c8-7f60-4e30-984d-962aaab91843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:00.842847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd86c8-7f60-4e30-984d-962aaab91843 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:00.843134 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd86c8-7f60-4e30-984d-962aaab91843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:00.843413 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd86c8-7f60-4e30-984d-962aaab91843 admin admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:00.844100 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cd86c8-7f60-4e30-984d-962aaab91843 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:00.846283 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cd86c8-7f60-4e30-984d-962aaab91843 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:00.893758 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e598528-4905-4234-ad62-3dc050e3cb28 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:00.897787 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daddf946-d3ae-4a32-9845-6887a147fa04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:00.898412 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daddf946-d3ae-4a32-9845-6887a147fa04 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:00.901915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daddf946-d3ae-4a32-9845-6887a147fa04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:00.901915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daddf946-d3ae-4a32-9845-6887a147fa04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:00.908557 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daddf946-d3ae-4a32-9845-6887a147fa04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:00.910061 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daddf946-d3ae-4a32-9845-6887a147fa04 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:00.919568 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1282/2583] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:38:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 15:38:00.979457 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1302/2584] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:38:00 2022] POST /identity/v3/users => generated 307 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:38:00.984041 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daddf946-d3ae-4a32-9845-6887a147fa04 service neutron] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:00.993735 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daddf946-d3ae-4a32-9845-6887a147fa04 service neutron] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:00.998573 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5e6e0-4969-4284-bf49-f47930b4e12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:00.999672 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5e6e0-4969-4284-bf49-f47930b4e12c admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:00.999959 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5e6e0-4969-4284-bf49-f47930b4e12c admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:01.000263 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5e6e0-4969-4284-bf49-f47930b4e12c admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:01.000948 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca5e6e0-4969-4284-bf49-f47930b4e12c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:01.002687 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca5e6e0-4969-4284-bf49-f47930b4e12c admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:01.013132 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1283/2585] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:38:01.017886 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd86c8-7f60-4e30-984d-962aaab91843 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:01.028985 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd86c8-7f60-4e30-984d-962aaab91843 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:01.034361 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf4784-662f-4535-b5ee-7bfafc66bb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:01.035059 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf4784-662f-4535-b5ee-7bfafc66bb87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:01.035346 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf4784-662f-4535-b5ee-7bfafc66bb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:01.035624 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf4784-662f-4535-b5ee-7bfafc66bb87 admin admin] PATH_INFO: `/v3/projects/9bb078d5cb21441aa9ad3e140cc9474e/users/096be6f330884448aaca46ad8182e6f5/roles/7c88c815db854954b516ab97cd471647` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:01.050928 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbf4784-662f-4535-b5ee-7bfafc66bb87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bb078d5cb21441aa9ad3e140cc9474e, user_id=096be6f330884448aaca46ad8182e6f5, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:01.053737 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbf4784-662f-4535-b5ee-7bfafc66bb87 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:01.139073 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1303/2586] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:38:01 2022] PUT /identity/v3/projects/9bb078d5cb21441aa9ad3e140cc9474e/users/096be6f330884448aaca46ad8182e6f5/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:38:01.143653 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5e6e0-4969-4284-bf49-f47930b4e12c admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:01.154029 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5e6e0-4969-4284-bf49-f47930b4e12c admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:01.181630 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:38:01.218854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff2df2-c0d6-4f9f-aedd-0866b487ea7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:01.218854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff2df2-c0d6-4f9f-aedd-0866b487ea7d admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:01.218854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff2df2-c0d6-4f9f-aedd-0866b487ea7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:01.218854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff2df2-c0d6-4f9f-aedd-0866b487ea7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:01.219238 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ff2df2-c0d6-4f9f-aedd-0866b487ea7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:01.220058 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ff2df2-c0d6-4f9f-aedd-0866b487ea7d admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:01.230016 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1284/2587] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:38:01.234888 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf4784-662f-4535-b5ee-7bfafc66bb87 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:01.246174 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf4784-662f-4535-b5ee-7bfafc66bb87 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:01.252269 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558478e7-a036-413b-8da4-f724c6556a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:01.253018 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558478e7-a036-413b-8da4-f724c6556a34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:01.253367 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558478e7-a036-413b-8da4-f724c6556a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:01.253709 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558478e7-a036-413b-8da4-f724c6556a34 admin admin] PATH_INFO: `/v3/projects/9bb078d5cb21441aa9ad3e140cc9474e/users/096be6f330884448aaca46ad8182e6f5/roles/7c88c815db854954b516ab97cd471647` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:01.256211 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558478e7-a036-413b-8da4-f724c6556a34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bb078d5cb21441aa9ad3e140cc9474e, user_id=096be6f330884448aaca46ad8182e6f5, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:01.259394 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558478e7-a036-413b-8da4-f724c6556a34 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:01.272179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1304/2588] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:38:01 2022] PUT /identity/v3/projects/9bb078d5cb21441aa9ad3e140cc9474e/users/096be6f330884448aaca46ad8182e6f5/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:38:01.283591 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57220de7-22bb-41d5-acee-fdf6782afe38 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:01.284133 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57220de7-22bb-41d5-acee-fdf6782afe38 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:01.284252 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57220de7-22bb-41d5-acee-fdf6782afe38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:01.313960 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-57220de7-22bb-41d5-acee-fdf6782afe38 None None] MFA Rules not processed for user `096be6f330884448aaca46ad8182e6f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:38:01.315213 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-57220de7-22bb-41d5-acee-fdf6782afe38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:01.384422 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1285/2589] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:38:01 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:38:01.395711 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558478e7-a036-413b-8da4-f724c6556a34 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:01.399748 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-558478e7-a036-413b-8da4-f724c6556a34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:01.458234 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558478e7-a036-413b-8da4-f724c6556a34 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:01.461934 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d115768-3263-4a11-8593-c2b14dc6c6b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:01.462558 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d115768-3263-4a11-8593-c2b14dc6c6b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:01.462889 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d115768-3263-4a11-8593-c2b14dc6c6b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:01.463185 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d115768-3263-4a11-8593-c2b14dc6c6b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:01.473052 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d115768-3263-4a11-8593-c2b14dc6c6b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:01.479937 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d115768-3263-4a11-8593-c2b14dc6c6b9 service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:01.496615 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1305/2590] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:38:01 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:38:01.720796 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57220de7-22bb-41d5-acee-fdf6782afe38 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:01.731307 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57220de7-22bb-41d5-acee-fdf6782afe38 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:01.736245 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343bcb78-623c-4c52-8fd1-c84fea40ff2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:01.736862 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343bcb78-623c-4c52-8fd1-c84fea40ff2d service cinder] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:01.737144 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343bcb78-623c-4c52-8fd1-c84fea40ff2d service cinder] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:01.737418 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343bcb78-623c-4c52-8fd1-c84fea40ff2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:01.794234 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343bcb78-623c-4c52-8fd1-c84fea40ff2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:01.795666 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343bcb78-623c-4c52-8fd1-c84fea40ff2d service cinder] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:01.805243 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1286/2591] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:38:01 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:38:06.469354 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8295833-81ae-4264-a25d-31422f561e53 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:06.469830 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8295833-81ae-4264-a25d-31422f561e53 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:06.469965 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8295833-81ae-4264-a25d-31422f561e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:06.504959 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-b8295833-81ae-4264-a25d-31422f561e53 None None] MFA Rules not processed for user `dd5c43d63a754eeaaba429f5c8153bba`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:38:06.506279 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-b8295833-81ae-4264-a25d-31422f561e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:06.549613 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:38:06.574879 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1306/2592] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:38:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:38:06.591760 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343bcb78-623c-4c52-8fd1-c84fea40ff2d service cinder] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:06.593276 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-343bcb78-623c-4c52-8fd1-c84fea40ff2d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:06.650545 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:38:06.683300 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343bcb78-623c-4c52-8fd1-c84fea40ff2d service cinder] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:06.691062 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec230ab9-4324-47d1-bb07-851d8ef3a18b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:06.691994 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec230ab9-4324-47d1-bb07-851d8ef3a18b service nova] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:06.692550 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec230ab9-4324-47d1-bb07-851d8ef3a18b service nova] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:06.692737 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec230ab9-4324-47d1-bb07-851d8ef3a18b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:06.702811 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec230ab9-4324-47d1-bb07-851d8ef3a18b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:06.704587 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec230ab9-4324-47d1-bb07-851d8ef3a18b service nova] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:06.718203 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1287/2593] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:38:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:38:07.058420 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8295833-81ae-4264-a25d-31422f561e53 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:07.123849 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8295833-81ae-4264-a25d-31422f561e53 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:07.129486 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01857ad2-8048-4e12-ba54-04dd9223f0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:07.130435 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01857ad2-8048-4e12-ba54-04dd9223f0b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:07.130575 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01857ad2-8048-4e12-ba54-04dd9223f0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:07.130857 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01857ad2-8048-4e12-ba54-04dd9223f0b6 admin admin] PATH_INFO: `/v3/users/d8c940c5c41d47c38ebd4dcb384ecf2c` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:07.132145 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01857ad2-8048-4e12-ba54-04dd9223f0b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8c940c5c41d47c38ebd4dcb384ecf2c)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:07.136177 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01857ad2-8048-4e12-ba54-04dd9223f0b6 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:07.351722 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-01857ad2-8048-4e12-ba54-04dd9223f0b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8c940c5c41d47c38ebd4dcb384ecf2c'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:07.352734 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-01857ad2-8048-4e12-ba54-04dd9223f0b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8c940c5c41d47c38ebd4dcb384ecf2c'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:07.366044 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-01857ad2-8048-4e12-ba54-04dd9223f0b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8c940c5c41d47c38ebd4dcb384ecf2c'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:07.419684 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-01857ad2-8048-4e12-ba54-04dd9223f0b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8c940c5c41d47c38ebd4dcb384ecf2c'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:07.428030 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-01857ad2-8048-4e12-ba54-04dd9223f0b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8c940c5c41d47c38ebd4dcb384ecf2c'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:07.434190 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1307/2594] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:38:07 2022] DELETE /identity/v3/users/d8c940c5c41d47c38ebd4dcb384ecf2c => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:38:07.438890 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd56b4f-afd0-46ef-ad39-ae66c3668107 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:07.439205 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd56b4f-afd0-46ef-ad39-ae66c3668107 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:07.439483 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd56b4f-afd0-46ef-ad39-ae66c3668107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:07.458857 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-8cd56b4f-afd0-46ef-ad39-ae66c3668107 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:38:07.460093 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-8cd56b4f-afd0-46ef-ad39-ae66c3668107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:07.527875 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1288/2595] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:38:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:38:07.538285 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01857ad2-8048-4e12-ba54-04dd9223f0b6 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:07.548474 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-01857ad2-8048-4e12-ba54-04dd9223f0b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:07.617995 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01857ad2-8048-4e12-ba54-04dd9223f0b6 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:07.622849 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db85aaf0-61a0-41ab-a248-b4a4224374bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:07.623518 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85aaf0-61a0-41ab-a248-b4a4224374bf service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:07.623782 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85aaf0-61a0-41ab-a248-b4a4224374bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:07.624246 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85aaf0-61a0-41ab-a248-b4a4224374bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:07.635603 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db85aaf0-61a0-41ab-a248-b4a4224374bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:07.637120 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db85aaf0-61a0-41ab-a248-b4a4224374bf service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:07.648197 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1308/2596] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:38:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 15:38:08.194792 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd56b4f-afd0-46ef-ad39-ae66c3668107 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:08.196477 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-8cd56b4f-afd0-46ef-ad39-ae66c3668107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:08.211455 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd56b4f-afd0-46ef-ad39-ae66c3668107 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:08.226549 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547565c-bc12-40b3-b57f-f6f5d4644268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:08.227219 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547565c-bc12-40b3-b57f-f6f5d4644268 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:08.227508 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547565c-bc12-40b3-b57f-f6f5d4644268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:08.228176 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547565c-bc12-40b3-b57f-f6f5d4644268 admin admin] PATH_INFO: `/v3/projects/423c220e15704c81afdf783e5bf70699` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:08.231733 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2547565c-bc12-40b3-b57f-f6f5d4644268 admin admin] RBAC: Authorizing `identity:delete_project(project_id=423c220e15704c81afdf783e5bf70699)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:08.233871 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2547565c-bc12-40b3-b57f-f6f5d4644268 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:08.367010 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-2547565c-bc12-40b3-b57f-f6f5d4644268 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '423c220e15704c81afdf783e5bf70699', 'request_id': 'req-2547565c-bc12-40b3-b57f-f6f5d4644268'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:08.374717 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-2547565c-bc12-40b3-b57f-f6f5d4644268 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '423c220e15704c81afdf783e5bf70699', 'request_id': 'req-2547565c-bc12-40b3-b57f-f6f5d4644268'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:08.442302 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-2547565c-bc12-40b3-b57f-f6f5d4644268 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '423c220e15704c81afdf783e5bf70699', 'request_id': 'req-2547565c-bc12-40b3-b57f-f6f5d4644268'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:08.455502 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-2547565c-bc12-40b3-b57f-f6f5d4644268 admin admin] The token cache is being invalidate because project 423c220e15704c81afdf783e5bf70699 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:38:08.459621 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-2547565c-bc12-40b3-b57f-f6f5d4644268 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:08.460543 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-2547565c-bc12-40b3-b57f-f6f5d4644268 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:08.465891 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1289/2597] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:38:08 2022] DELETE /identity/v3/projects/423c220e15704c81afdf783e5bf70699 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:38:08.467171 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db85aaf0-61a0-41ab-a248-b4a4224374bf service neutron] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:08.472964 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:38:08.478617 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db85aaf0-61a0-41ab-a248-b4a4224374bf service neutron] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:08.543733 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953705d2-dfeb-4a19-918b-506772f658cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:08.544436 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953705d2-dfeb-4a19-918b-506772f658cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:08.545039 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953705d2-dfeb-4a19-918b-506772f658cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:08.545039 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953705d2-dfeb-4a19-918b-506772f658cd admin admin] PATH_INFO: `/v3/users/096be6f330884448aaca46ad8182e6f5` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:08.546173 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953705d2-dfeb-4a19-918b-506772f658cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=096be6f330884448aaca46ad8182e6f5)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:08.548159 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953705d2-dfeb-4a19-918b-506772f658cd admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:08.777352 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-953705d2-dfeb-4a19-918b-506772f658cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '096be6f330884448aaca46ad8182e6f5'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:08.778008 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-953705d2-dfeb-4a19-918b-506772f658cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '096be6f330884448aaca46ad8182e6f5'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:08.790576 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-953705d2-dfeb-4a19-918b-506772f658cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '096be6f330884448aaca46ad8182e6f5'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:08.888265 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-953705d2-dfeb-4a19-918b-506772f658cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '096be6f330884448aaca46ad8182e6f5'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:08.896320 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-953705d2-dfeb-4a19-918b-506772f658cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '096be6f330884448aaca46ad8182e6f5'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:08.903126 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1309/2598] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:38:08 2022] DELETE /identity/v3/users/096be6f330884448aaca46ad8182e6f5 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:38:09.352528 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce0e10-49e0-47da-b712-f43f69151b9d None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:09.353375 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce0e10-49e0-47da-b712-f43f69151b9d None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:09.353375 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce0e10-49e0-47da-b712-f43f69151b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:09.373356 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-f4ce0e10-49e0-47da-b712-f43f69151b9d None None] MFA Rules not processed for user `dd5c43d63a754eeaaba429f5c8153bba`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:38:09.374729 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-f4ce0e10-49e0-47da-b712-f43f69151b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:09.382407 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953705d2-dfeb-4a19-918b-506772f658cd admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:09.382407 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-953705d2-dfeb-4a19-918b-506772f658cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:09.440468 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:38:09.448517 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1290/2599] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:38:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:38:09.472919 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953705d2-dfeb-4a19-918b-506772f658cd admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:09.477771 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a5bf72-df42-4163-9da9-ef6d1d855bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:09.478613 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5bf72-df42-4163-9da9-ef6d1d855bf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:09.479142 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5bf72-df42-4163-9da9-ef6d1d855bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:09.479625 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5bf72-df42-4163-9da9-ef6d1d855bf2 admin admin] PATH_INFO: `/v3/projects/9bb078d5cb21441aa9ad3e140cc9474e` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:09.481116 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a5bf72-df42-4163-9da9-ef6d1d855bf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bb078d5cb21441aa9ad3e140cc9474e)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:09.483535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a5bf72-df42-4163-9da9-ef6d1d855bf2 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:09.525691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:38:09.641587 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-a5a5bf72-df42-4163-9da9-ef6d1d855bf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bb078d5cb21441aa9ad3e140cc9474e', 'request_id': 'req-a5a5bf72-df42-4163-9da9-ef6d1d855bf2'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:09.649677 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-a5a5bf72-df42-4163-9da9-ef6d1d855bf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bb078d5cb21441aa9ad3e140cc9474e', 'request_id': 'req-a5a5bf72-df42-4163-9da9-ef6d1d855bf2'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:09.657561 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-a5a5bf72-df42-4163-9da9-ef6d1d855bf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bb078d5cb21441aa9ad3e140cc9474e', 'request_id': 'req-a5a5bf72-df42-4163-9da9-ef6d1d855bf2'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:09.730230 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-a5a5bf72-df42-4163-9da9-ef6d1d855bf2 admin admin] The token cache is being invalidate because project 9bb078d5cb21441aa9ad3e140cc9474e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:38:09.730899 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-a5a5bf72-df42-4163-9da9-ef6d1d855bf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:09.731730 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-a5a5bf72-df42-4163-9da9-ef6d1d855bf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:09.735038 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1310/2600] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:38:09 2022] DELETE /identity/v3/projects/9bb078d5cb21441aa9ad3e140cc9474e => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:38:09.741688 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22ad92-0292-4b09-ac3b-c85679b0dbc0 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:09.741983 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22ad92-0292-4b09-ac3b-c85679b0dbc0 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:09.742263 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22ad92-0292-4b09-ac3b-c85679b0dbc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:09.765695 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-de22ad92-0292-4b09-ac3b-c85679b0dbc0 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:38:09.767004 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-de22ad92-0292-4b09-ac3b-c85679b0dbc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:09.860239 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1291/2601] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:38:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:38:09.868426 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dcaeb3-adc8-431b-a4c0-7f79faa740ac None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:09.868795 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dcaeb3-adc8-431b-a4c0-7f79faa740ac None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:09.869690 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dcaeb3-adc8-431b-a4c0-7f79faa740ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:09.892401 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-31dcaeb3-adc8-431b-a4c0-7f79faa740ac None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:38:09.894142 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-31dcaeb3-adc8-431b-a4c0-7f79faa740ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:09.902787 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1311/2602] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:38:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:38:09.907396 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de22ad92-0292-4b09-ac3b-c85679b0dbc0 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:09.917557 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de22ad92-0292-4b09-ac3b-c85679b0dbc0 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:09.922855 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501bab4e-300f-445e-bce6-737056f1f15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:09.923641 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501bab4e-300f-445e-bce6-737056f1f15e admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:09.923941 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501bab4e-300f-445e-bce6-737056f1f15e admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:09.924372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501bab4e-300f-445e-bce6-737056f1f15e admin admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:09.925540 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501bab4e-300f-445e-bce6-737056f1f15e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:38:09.925540 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501bab4e-300f-445e-bce6-737056f1f15e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:09.927695 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501bab4e-300f-445e-bce6-737056f1f15e admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:09.941972 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1292/2603] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:38:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:38:09.946435 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dcaeb3-adc8-431b-a4c0-7f79faa740ac None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:09.957550 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dcaeb3-adc8-431b-a4c0-7f79faa740ac None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:09.964745 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148b0372-ab28-47db-9606-69eeeb109b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:09.965588 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b0372-ab28-47db-9606-69eeeb109b61 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:09.965992 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b0372-ab28-47db-9606-69eeeb109b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:09.966421 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b0372-ab28-47db-9606-69eeeb109b61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:09.967203 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148b0372-ab28-47db-9606-69eeeb109b61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:09.970062 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148b0372-ab28-47db-9606-69eeeb109b61 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:10.057530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1312/2604] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:38:09 2022] POST /identity/v3/projects => generated 369 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:38:10.058392 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501bab4e-300f-445e-bce6-737056f1f15e admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:10.070040 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501bab4e-300f-445e-bce6-737056f1f15e admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:10.144837 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d552b0-6c80-456c-8ffa-abde57531698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:10.145678 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d552b0-6c80-456c-8ffa-abde57531698 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:10.146180 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d552b0-6c80-456c-8ffa-abde57531698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:10.146475 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d552b0-6c80-456c-8ffa-abde57531698 admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:10.147260 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d552b0-6c80-456c-8ffa-abde57531698 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:10.149756 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d552b0-6c80-456c-8ffa-abde57531698 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:10.300536 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1293/2605] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:38:10 2022] POST /identity/v3/users => generated 285 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:38:10.304453 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148b0372-ab28-47db-9606-69eeeb109b61 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:10.326817 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148b0372-ab28-47db-9606-69eeeb109b61 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:10.332802 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878fc451-dab3-41f5-b56b-ac710c97c36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:10.333440 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878fc451-dab3-41f5-b56b-ac710c97c36e admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:10.337928 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878fc451-dab3-41f5-b56b-ac710c97c36e admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:10.337928 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878fc451-dab3-41f5-b56b-ac710c97c36e admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:10.337928 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878fc451-dab3-41f5-b56b-ac710c97c36e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:10.337928 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878fc451-dab3-41f5-b56b-ac710c97c36e admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:10.354742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1313/2606] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:38:10 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:38:10.359310 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d552b0-6c80-456c-8ffa-abde57531698 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:10.371980 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d552b0-6c80-456c-8ffa-abde57531698 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:10.379158 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb76ae-8caa-4f01-8d73-a74f0a48e619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:10.380011 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb76ae-8caa-4f01-8d73-a74f0a48e619 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:10.380288 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb76ae-8caa-4f01-8d73-a74f0a48e619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:10.380571 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb76ae-8caa-4f01-8d73-a74f0a48e619 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:10.382473 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eb76ae-8caa-4f01-8d73-a74f0a48e619 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:10.385896 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eb76ae-8caa-4f01-8d73-a74f0a48e619 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:10.402941 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1294/2607] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:38:10 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:38:10.404087 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878fc451-dab3-41f5-b56b-ac710c97c36e admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:10.415024 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878fc451-dab3-41f5-b56b-ac710c97c36e admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:10.419794 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b526275d-e26d-4ee0-8126-b86a0ffaca3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:10.420566 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526275d-e26d-4ee0-8126-b86a0ffaca3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:10.420948 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526275d-e26d-4ee0-8126-b86a0ffaca3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:10.421237 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526275d-e26d-4ee0-8126-b86a0ffaca3d admin admin] PATH_INFO: `/v3/projects/23674d5fe9474fa5a27795efc6dc9f76/users/ace2f2d6edd7440e8d8423405d8738e5/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:10.436138 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b526275d-e26d-4ee0-8126-b86a0ffaca3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=23674d5fe9474fa5a27795efc6dc9f76, user_id=ace2f2d6edd7440e8d8423405d8738e5, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:10.439334 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b526275d-e26d-4ee0-8126-b86a0ffaca3d admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:10.551938 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1314/2608] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:38:10 2022] PUT /identity/v3/projects/23674d5fe9474fa5a27795efc6dc9f76/users/ace2f2d6edd7440e8d8423405d8738e5/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:38:10.562969 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778dd9b5-10b4-4a6d-b170-f9f8132d0d81 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:10.563259 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778dd9b5-10b4-4a6d-b170-f9f8132d0d81 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:10.563536 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778dd9b5-10b4-4a6d-b170-f9f8132d0d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:10.592315 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-778dd9b5-10b4-4a6d-b170-f9f8132d0d81 None None] MFA Rules not processed for user `ace2f2d6edd7440e8d8423405d8738e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:38:10.593541 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-778dd9b5-10b4-4a6d-b170-f9f8132d0d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:10.651054 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:38:10.660113 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1295/2609] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:38:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:38:10.672001 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b526275d-e26d-4ee0-8126-b86a0ffaca3d admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:10.673711 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-b526275d-e26d-4ee0-8126-b86a0ffaca3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:10.731651 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b526275d-e26d-4ee0-8126-b86a0ffaca3d admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:10.736547 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a5446e-6ed1-46db-89b8-712faedf224a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c386fa0483cf42efaddbc2395f6d1fce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:10.737169 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a5446e-6ed1-46db-89b8-712faedf224a service glance] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:10.737450 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a5446e-6ed1-46db-89b8-712faedf224a service glance] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:10.737795 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a5446e-6ed1-46db-89b8-712faedf224a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:10.746516 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a5446e-6ed1-46db-89b8-712faedf224a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:10.747939 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a5446e-6ed1-46db-89b8-712faedf224a service glance] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:10.757600 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1315/2610] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:38:10 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 15:38:16.637283 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778dd9b5-10b4-4a6d-b170-f9f8132d0d81 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:16.638703 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-778dd9b5-10b4-4a6d-b170-f9f8132d0d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:16.705332 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778dd9b5-10b4-4a6d-b170-f9f8132d0d81 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:16.710089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6709e6-9dee-4175-a285-6acdf921aa29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:16.710723 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6709e6-9dee-4175-a285-6acdf921aa29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:16.711014 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6709e6-9dee-4175-a285-6acdf921aa29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:16.711291 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6709e6-9dee-4175-a285-6acdf921aa29 admin admin] PATH_INFO: `/v3/users/6b440e7d2c4d443dbb35a1378131f55d` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:16.712482 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6709e6-9dee-4175-a285-6acdf921aa29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b440e7d2c4d443dbb35a1378131f55d)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:16.714422 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6709e6-9dee-4175-a285-6acdf921aa29 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:16.908420 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-6d6709e6-9dee-4175-a285-6acdf921aa29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b440e7d2c4d443dbb35a1378131f55d'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:16.915302 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-6d6709e6-9dee-4175-a285-6acdf921aa29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b440e7d2c4d443dbb35a1378131f55d'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:16.915963 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-6d6709e6-9dee-4175-a285-6acdf921aa29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b440e7d2c4d443dbb35a1378131f55d'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:16.927916 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-6d6709e6-9dee-4175-a285-6acdf921aa29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b440e7d2c4d443dbb35a1378131f55d'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:16.928581 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-6d6709e6-9dee-4175-a285-6acdf921aa29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b440e7d2c4d443dbb35a1378131f55d'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:17.008564 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1296/2611] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:38:16 2022] DELETE /identity/v3/users/6b440e7d2c4d443dbb35a1378131f55d => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:38:17.013226 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c7d12-7ddc-47fd-b8ec-645e2818393f None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:17.013979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c7d12-7ddc-47fd-b8ec-645e2818393f None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:17.013979 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c7d12-7ddc-47fd-b8ec-645e2818393f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:17.034081 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-f56c7d12-7ddc-47fd-b8ec-645e2818393f None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:38:17.035298 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-f56c7d12-7ddc-47fd-b8ec-645e2818393f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:17.083033 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:38:17.100761 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1316/2612] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:38:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:38:17.112263 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6709e6-9dee-4175-a285-6acdf921aa29 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:17.113676 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-6d6709e6-9dee-4175-a285-6acdf921aa29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:17.171637 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6709e6-9dee-4175-a285-6acdf921aa29 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:17.175414 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5698a7-3201-45c6-ba78-689c24e1c1b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:17.176145 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5698a7-3201-45c6-ba78-689c24e1c1b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:17.176533 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5698a7-3201-45c6-ba78-689c24e1c1b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:17.176815 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5698a7-3201-45c6-ba78-689c24e1c1b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:17.187465 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5698a7-3201-45c6-ba78-689c24e1c1b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:17.188934 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5698a7-3201-45c6-ba78-689c24e1c1b5 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:17.198437 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1297/2613] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:38:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:38:17.605053 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c7d12-7ddc-47fd-b8ec-645e2818393f None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:17.606819 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-f56c7d12-7ddc-47fd-b8ec-645e2818393f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:17.619946 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c7d12-7ddc-47fd-b8ec-645e2818393f None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:17.625694 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c601e9d-a16c-4f30-9ed1-f66c408aa85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:17.626912 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c601e9d-a16c-4f30-9ed1-f66c408aa85b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:17.627204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c601e9d-a16c-4f30-9ed1-f66c408aa85b admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:17.627505 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c601e9d-a16c-4f30-9ed1-f66c408aa85b admin admin] PATH_INFO: `/v3/projects/0b8c2f1cc2f4495da1f2312f76dc3614` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:17.628745 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c601e9d-a16c-4f30-9ed1-f66c408aa85b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b8c2f1cc2f4495da1f2312f76dc3614)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:17.630658 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c601e9d-a16c-4f30-9ed1-f66c408aa85b admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:17.730443 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-4c601e9d-a16c-4f30-9ed1-f66c408aa85b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b8c2f1cc2f4495da1f2312f76dc3614', 'request_id': 'req-4c601e9d-a16c-4f30-9ed1-f66c408aa85b'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:17.738367 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-4c601e9d-a16c-4f30-9ed1-f66c408aa85b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b8c2f1cc2f4495da1f2312f76dc3614', 'request_id': 'req-4c601e9d-a16c-4f30-9ed1-f66c408aa85b'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:17.745579 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-4c601e9d-a16c-4f30-9ed1-f66c408aa85b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b8c2f1cc2f4495da1f2312f76dc3614', 'request_id': 'req-4c601e9d-a16c-4f30-9ed1-f66c408aa85b'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:17.807976 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-4c601e9d-a16c-4f30-9ed1-f66c408aa85b admin admin] The token cache is being invalidate because project 0b8c2f1cc2f4495da1f2312f76dc3614 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:38:17.808441 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-4c601e9d-a16c-4f30-9ed1-f66c408aa85b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:17.809093 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-4c601e9d-a16c-4f30-9ed1-f66c408aa85b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:17.810597 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1317/2614] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:38:17 2022] DELETE /identity/v3/projects/0b8c2f1cc2f4495da1f2312f76dc3614 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:38:17.815480 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0c2a2-f9fd-49dc-9024-0038e6636367 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:17.815963 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0c2a2-f9fd-49dc-9024-0038e6636367 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:17.816096 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0c2a2-f9fd-49dc-9024-0038e6636367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:17.836180 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-e3c0c2a2-f9fd-49dc-9024-0038e6636367 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:38:17.837400 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-e3c0c2a2-f9fd-49dc-9024-0038e6636367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:17.850524 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:38:17.900752 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1298/2615] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:38:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:38:17.967268 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab334b-c9e7-4f69-ae6d-bfd435759006 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:17.967553 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab334b-c9e7-4f69-ae6d-bfd435759006 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:17.968412 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab334b-c9e7-4f69-ae6d-bfd435759006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:17.988375 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-f8ab334b-c9e7-4f69-ae6d-bfd435759006 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:38:17.989509 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-f8ab334b-c9e7-4f69-ae6d-bfd435759006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:17.997352 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1318/2616] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:38:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:38:18.001559 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0c2a2-f9fd-49dc-9024-0038e6636367 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:18.011396 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0c2a2-f9fd-49dc-9024-0038e6636367 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:18.016161 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab1df42-4301-493f-8868-067597b31445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:18.016771 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab1df42-4301-493f-8868-067597b31445 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:18.017058 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab1df42-4301-493f-8868-067597b31445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:18.017336 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab1df42-4301-493f-8868-067597b31445 admin admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:18.018030 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab1df42-4301-493f-8868-067597b31445 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:38:18.018340 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab1df42-4301-493f-8868-067597b31445 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:18.019920 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab1df42-4301-493f-8868-067597b31445 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:18.032239 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1299/2617] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:38:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:38:18.036507 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ab334b-c9e7-4f69-ae6d-bfd435759006 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:18.045405 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ab334b-c9e7-4f69-ae6d-bfd435759006 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:18.050593 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb00826-62bd-4969-a039-0d0845f3e227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:18.051186 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb00826-62bd-4969-a039-0d0845f3e227 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:18.051474 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb00826-62bd-4969-a039-0d0845f3e227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:18.051741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb00826-62bd-4969-a039-0d0845f3e227 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:18.052425 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb00826-62bd-4969-a039-0d0845f3e227 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:18.054301 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb00826-62bd-4969-a039-0d0845f3e227 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:18.151454 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1319/2618] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:38:18 2022] POST /identity/v3/projects => generated 375 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:38:18.152869 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab1df42-4301-493f-8868-067597b31445 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:18.163424 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab1df42-4301-493f-8868-067597b31445 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:18.225234 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49226b5-488a-4cd0-a61f-a8b35ca38b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:18.225851 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49226b5-488a-4cd0-a61f-a8b35ca38b9f admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:18.226134 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49226b5-488a-4cd0-a61f-a8b35ca38b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:18.226598 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49226b5-488a-4cd0-a61f-a8b35ca38b9f admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:18.227276 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49226b5-488a-4cd0-a61f-a8b35ca38b9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:18.229222 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49226b5-488a-4cd0-a61f-a8b35ca38b9f admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:18.307740 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1300/2619] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:38:18 2022] POST /identity/v3/users => generated 288 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:38:18.312744 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb00826-62bd-4969-a039-0d0845f3e227 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:18.323195 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb00826-62bd-4969-a039-0d0845f3e227 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:18.327890 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e37d5-24e9-43b6-8232-fb4bdfe3a02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:18.332634 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e37d5-24e9-43b6-8232-fb4bdfe3a02b admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:18.332634 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e37d5-24e9-43b6-8232-fb4bdfe3a02b admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:18.332634 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e37d5-24e9-43b6-8232-fb4bdfe3a02b admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:18.332634 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e37d5-24e9-43b6-8232-fb4bdfe3a02b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:18.332634 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e37d5-24e9-43b6-8232-fb4bdfe3a02b admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:18.341632 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1320/2620] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:38:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:38:18.345561 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49226b5-488a-4cd0-a61f-a8b35ca38b9f admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:18.355495 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49226b5-488a-4cd0-a61f-a8b35ca38b9f admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:18.360605 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ff61fe-a159-4fc1-b3a6-7195e84a3798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:18.361206 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ff61fe-a159-4fc1-b3a6-7195e84a3798 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:18.361487 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ff61fe-a159-4fc1-b3a6-7195e84a3798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:18.361768 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ff61fe-a159-4fc1-b3a6-7195e84a3798 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:18.362453 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ff61fe-a159-4fc1-b3a6-7195e84a3798 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:18.364099 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ff61fe-a159-4fc1-b3a6-7195e84a3798 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:18.375905 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1301/2621] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:38:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:38:18.381111 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e37d5-24e9-43b6-8232-fb4bdfe3a02b admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:18.394114 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e37d5-24e9-43b6-8232-fb4bdfe3a02b admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:18.399324 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f401c247-0143-4a35-8c21-c8b64ba842cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:18.400025 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401c247-0143-4a35-8c21-c8b64ba842cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:18.400363 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401c247-0143-4a35-8c21-c8b64ba842cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:18.400674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401c247-0143-4a35-8c21-c8b64ba842cb admin admin] PATH_INFO: `/v3/projects/acd3963551a44ea8a6f9f5829f7d2332/users/22f4a016c52647db95dfdd2c29c7bec3/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:18.416390 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f401c247-0143-4a35-8c21-c8b64ba842cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=acd3963551a44ea8a6f9f5829f7d2332, user_id=22f4a016c52647db95dfdd2c29c7bec3, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:18.419069 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f401c247-0143-4a35-8c21-c8b64ba842cb admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:18.473761 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1321/2622] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:38:18 2022] PUT /identity/v3/projects/acd3963551a44ea8a6f9f5829f7d2332/users/22f4a016c52647db95dfdd2c29c7bec3/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:38:18.484721 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebc8fd-a5be-49f5-8d90-f1ec033eab46 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:18.485092 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebc8fd-a5be-49f5-8d90-f1ec033eab46 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:18.485373 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebc8fd-a5be-49f5-8d90-f1ec033eab46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:18.514016 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-08ebc8fd-a5be-49f5-8d90-f1ec033eab46 None None] MFA Rules not processed for user `22f4a016c52647db95dfdd2c29c7bec3`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:38:18.515660 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-08ebc8fd-a5be-49f5-8d90-f1ec033eab46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:18.583986 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1302/2623] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:38:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:38:18.595246 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f401c247-0143-4a35-8c21-c8b64ba842cb admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:18.597001 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-f401c247-0143-4a35-8c21-c8b64ba842cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:18.662457 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f401c247-0143-4a35-8c21-c8b64ba842cb admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:18.667543 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b36cda8-383f-46af-99cc-79dd09661fc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:18.668563 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b36cda8-383f-46af-99cc-79dd09661fc7 service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:18.668863 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b36cda8-383f-46af-99cc-79dd09661fc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:18.669287 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b36cda8-383f-46af-99cc-79dd09661fc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:18.687584 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b36cda8-383f-46af-99cc-79dd09661fc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:18.689075 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b36cda8-383f-46af-99cc-79dd09661fc7 service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:18.698604 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1322/2624] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:38:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:38:20.380169 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5a53a-10f1-4290-95ff-1d845f341711 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:20.380645 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5a53a-10f1-4290-95ff-1d845f341711 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:20.380751 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5a53a-10f1-4290-95ff-1d845f341711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:20.401245 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-e8b5a53a-10f1-4290-95ff-1d845f341711 None None] MFA Rules not processed for user `22f4a016c52647db95dfdd2c29c7bec3`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:38:20.402497 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-e8b5a53a-10f1-4290-95ff-1d845f341711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:20.410141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1303/2625] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:38:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:38:21.725449 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b36cda8-383f-46af-99cc-79dd09661fc7 service cinder] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:21.727417 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-9b36cda8-383f-46af-99cc-79dd09661fc7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:21.739558 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b36cda8-383f-46af-99cc-79dd09661fc7 service cinder] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:21.743182 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c190ce1-618a-4fbc-b543-df87548f0613 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:21.743789 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c190ce1-618a-4fbc-b543-df87548f0613 service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:21.744099 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c190ce1-618a-4fbc-b543-df87548f0613 service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:21.744382 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c190ce1-618a-4fbc-b543-df87548f0613 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:21.747121 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-9c190ce1-618a-4fbc-b543-df87548f0613 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:21.756051 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:38:21.804113 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c190ce1-618a-4fbc-b543-df87548f0613 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:21.805554 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c190ce1-618a-4fbc-b543-df87548f0613 service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:21.815924 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1323/2626] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:38:21 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:38:30.394368 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=94,keystone:INSERT=3 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:38:31.372949 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8323a-83ee-44cd-89b7-3be75ec4f065 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:31.373353 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8323a-83ee-44cd-89b7-3be75ec4f065 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:31.373529 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8323a-83ee-44cd-89b7-3be75ec4f065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:31.393139 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-c8f8323a-83ee-44cd-89b7-3be75ec4f065 None None] MFA Rules not processed for user `1782b4e84c9f484297e17c248738105c`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:38:31.398875 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-c8f8323a-83ee-44cd-89b7-3be75ec4f065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:31.401565 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1304/2627] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:38:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:38:32.014357 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=22 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:38:41.388020 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=3 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:38:46.253068 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e879b3e8-7930-4582-8447-f0d20a8d9302 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:46.253552 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e879b3e8-7930-4582-8447-f0d20a8d9302 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:46.253664 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e879b3e8-7930-4582-8447-f0d20a8d9302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:46.273419 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-e879b3e8-7930-4582-8447-f0d20a8d9302 None None] MFA Rules not processed for user `1654fb6a81e14d6b99ab495bb6d029d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:38:46.277884 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-e879b3e8-7930-4582-8447-f0d20a8d9302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:46.328133 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1324/2628] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 5 15:38:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:38:46.486988 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f8323a-83ee-44cd-89b7-3be75ec4f065 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:46.488717 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-c8f8323a-83ee-44cd-89b7-3be75ec4f065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:46.561280 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f8323a-83ee-44cd-89b7-3be75ec4f065 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:46.566958 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41f6d2-9568-471f-83c5-f60390dec48b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85154ce7446245738edcc8702dbf6861', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:46.567580 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41f6d2-9568-471f-83c5-f60390dec48b service swift] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:46.567866 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41f6d2-9568-471f-83c5-f60390dec48b service swift] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:46.568179 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41f6d2-9568-471f-83c5-f60390dec48b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:46.576371 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d41f6d2-9568-471f-83c5-f60390dec48b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:46.577955 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d41f6d2-9568-471f-83c5-f60390dec48b service swift] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:46.587267 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1305/2629] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 5 15:38:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 15:38:47.492664 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95f83c-2ac9-4d41-ba5b-e8faf9a8fd70 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:47.496237 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95f83c-2ac9-4d41-ba5b-e8faf9a8fd70 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:47.496237 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95f83c-2ac9-4d41-ba5b-e8faf9a8fd70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:47.515190 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-8f95f83c-2ac9-4d41-ba5b-e8faf9a8fd70 None None] MFA Rules not processed for user `1654fb6a81e14d6b99ab495bb6d029d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:38:47.515517 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-8f95f83c-2ac9-4d41-ba5b-e8faf9a8fd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:47.522834 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1325/2630] 10.222.0.40 () {58 vars in 967 bytes} [Fri Aug 5 15:38:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:38:47.535116 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41f6d2-9568-471f-83c5-f60390dec48b service swift] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:47.545714 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41f6d2-9568-471f-83c5-f60390dec48b service swift] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:47.551260 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f85bf-8d4b-4f39-ae7e-093404ed32cc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85154ce7446245738edcc8702dbf6861', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:47.551865 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f85bf-8d4b-4f39-ae7e-093404ed32cc service swift] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:47.552194 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f85bf-8d4b-4f39-ae7e-093404ed32cc service swift] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:47.552469 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f85bf-8d4b-4f39-ae7e-093404ed32cc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:47.560459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12f85bf-8d4b-4f39-ae7e-093404ed32cc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:47.562148 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12f85bf-8d4b-4f39-ae7e-093404ed32cc service swift] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:47.570590 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1306/2631] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 5 15:38:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 15:38:48.162917 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52af96d-5ea3-4801-8d37-5019ad181193 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:48.163210 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52af96d-5ea3-4801-8d37-5019ad181193 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:48.163499 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52af96d-5ea3-4801-8d37-5019ad181193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:48.183958 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-d52af96d-5ea3-4801-8d37-5019ad181193 None None] MFA Rules not processed for user `1654fb6a81e14d6b99ab495bb6d029d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:38:48.185224 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-d52af96d-5ea3-4801-8d37-5019ad181193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:48.192693 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1326/2632] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 5 15:38:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:38:48.204965 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f85bf-8d4b-4f39-ae7e-093404ed32cc service swift] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:48.215524 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f85bf-8d4b-4f39-ae7e-093404ed32cc service swift] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:48.221176 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c2d28-8e6f-4ee2-aa40-c16bc29bb588 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85154ce7446245738edcc8702dbf6861', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:48.221777 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c2d28-8e6f-4ee2-aa40-c16bc29bb588 service swift] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:48.222065 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c2d28-8e6f-4ee2-aa40-c16bc29bb588 service swift] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:48.222344 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c2d28-8e6f-4ee2-aa40-c16bc29bb588 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:48.230255 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69c2d28-8e6f-4ee2-aa40-c16bc29bb588 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:48.231826 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69c2d28-8e6f-4ee2-aa40-c16bc29bb588 service swift] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:48.239916 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1307/2633] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 5 15:38:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 15:38:48.883742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad400c-d67c-4c8a-89e2-c085912bffea None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:48.883742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad400c-d67c-4c8a-89e2-c085912bffea None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:48.883742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad400c-d67c-4c8a-89e2-c085912bffea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:48.901036 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-ffad400c-d67c-4c8a-89e2-c085912bffea None None] MFA Rules not processed for user `1654fb6a81e14d6b99ab495bb6d029d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:38:48.902288 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-ffad400c-d67c-4c8a-89e2-c085912bffea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:48.909672 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1327/2634] 10.222.0.40 () {58 vars in 967 bytes} [Fri Aug 5 15:38:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:38:48.922971 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c2d28-8e6f-4ee2-aa40-c16bc29bb588 service swift] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:48.934570 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c2d28-8e6f-4ee2-aa40-c16bc29bb588 service swift] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:48.940773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daff5756-2d90-476a-8317-77eb413a45d0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85154ce7446245738edcc8702dbf6861', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:48.941399 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff5756-2d90-476a-8317-77eb413a45d0 service swift] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:48.941684 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff5756-2d90-476a-8317-77eb413a45d0 service swift] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:48.941966 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff5756-2d90-476a-8317-77eb413a45d0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:48.950901 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daff5756-2d90-476a-8317-77eb413a45d0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:48.952522 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daff5756-2d90-476a-8317-77eb413a45d0 service swift] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:48.962115 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1308/2635] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 5 15:38:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 05 15:38:49.152643 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad400c-d67c-4c8a-89e2-c085912bffea None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:49.154061 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-ffad400c-d67c-4c8a-89e2-c085912bffea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:49.226654 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad400c-d67c-4c8a-89e2-c085912bffea None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:49.231607 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519eefb0-fa4a-4d4f-939b-a0614573f61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:49.232515 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519eefb0-fa4a-4d4f-939b-a0614573f61d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:49.232803 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519eefb0-fa4a-4d4f-939b-a0614573f61d admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:49.233083 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519eefb0-fa4a-4d4f-939b-a0614573f61d admin admin] PATH_INFO: `/v3/users/ace2f2d6edd7440e8d8423405d8738e5` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:49.234361 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519eefb0-fa4a-4d4f-939b-a0614573f61d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ace2f2d6edd7440e8d8423405d8738e5)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:49.236814 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519eefb0-fa4a-4d4f-939b-a0614573f61d admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:49.452168 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-519eefb0-fa4a-4d4f-939b-a0614573f61d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ace2f2d6edd7440e8d8423405d8738e5'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:49.452870 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-519eefb0-fa4a-4d4f-939b-a0614573f61d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ace2f2d6edd7440e8d8423405d8738e5'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:49.466596 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-519eefb0-fa4a-4d4f-939b-a0614573f61d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ace2f2d6edd7440e8d8423405d8738e5'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:49.546376 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-519eefb0-fa4a-4d4f-939b-a0614573f61d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ace2f2d6edd7440e8d8423405d8738e5'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:49.558489 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-519eefb0-fa4a-4d4f-939b-a0614573f61d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ace2f2d6edd7440e8d8423405d8738e5'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:49.564844 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1328/2636] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:38:49 2022] DELETE /identity/v3/users/ace2f2d6edd7440e8d8423405d8738e5 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:38:49.569216 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b05aa8-23b3-4d92-8b48-c017845c7602 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:49.573419 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b05aa8-23b3-4d92-8b48-c017845c7602 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:49.573419 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b05aa8-23b3-4d92-8b48-c017845c7602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:49.592996 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-d6b05aa8-23b3-4d92-8b48-c017845c7602 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:38:49.594234 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-d6b05aa8-23b3-4d92-8b48-c017845c7602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:49.660910 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1309/2637] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:38:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:38:49.668494 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519eefb0-fa4a-4d4f-939b-a0614573f61d admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:49.670031 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-519eefb0-fa4a-4d4f-939b-a0614573f61d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:49.678484 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:38:49.729666 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519eefb0-fa4a-4d4f-939b-a0614573f61d admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:49.733518 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c8671c-9738-465b-aa35-5cb5cdb76a24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:49.734200 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8671c-9738-465b-aa35-5cb5cdb76a24 service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:49.734491 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8671c-9738-465b-aa35-5cb5cdb76a24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:49.734778 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8671c-9738-465b-aa35-5cb5cdb76a24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:49.744296 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c8671c-9738-465b-aa35-5cb5cdb76a24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:49.745738 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c8671c-9738-465b-aa35-5cb5cdb76a24 service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:49.756130 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1329/2638] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:38:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:38:50.177471 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b05aa8-23b3-4d92-8b48-c017845c7602 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:50.178915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-d6b05aa8-23b3-4d92-8b48-c017845c7602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:50.192471 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b05aa8-23b3-4d92-8b48-c017845c7602 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:50.211335 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6b7a23-7601-4e63-b942-505f3ce7b135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:50.212052 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b7a23-7601-4e63-b942-505f3ce7b135 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:50.212337 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b7a23-7601-4e63-b942-505f3ce7b135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:50.212590 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b7a23-7601-4e63-b942-505f3ce7b135 admin admin] PATH_INFO: `/v3/projects/23674d5fe9474fa5a27795efc6dc9f76` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:50.213879 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6b7a23-7601-4e63-b942-505f3ce7b135 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23674d5fe9474fa5a27795efc6dc9f76)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:50.215965 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6b7a23-7601-4e63-b942-505f3ce7b135 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:50.336644 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:38:50.339474 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-4f6b7a23-7601-4e63-b942-505f3ce7b135 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23674d5fe9474fa5a27795efc6dc9f76', 'request_id': 'req-4f6b7a23-7601-4e63-b942-505f3ce7b135'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:50.351406 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-4f6b7a23-7601-4e63-b942-505f3ce7b135 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23674d5fe9474fa5a27795efc6dc9f76', 'request_id': 'req-4f6b7a23-7601-4e63-b942-505f3ce7b135'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:50.432305 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-4f6b7a23-7601-4e63-b942-505f3ce7b135 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23674d5fe9474fa5a27795efc6dc9f76', 'request_id': 'req-4f6b7a23-7601-4e63-b942-505f3ce7b135'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:50.446184 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-4f6b7a23-7601-4e63-b942-505f3ce7b135 admin admin] The token cache is being invalidate because project 23674d5fe9474fa5a27795efc6dc9f76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:38:50.446717 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-4f6b7a23-7601-4e63-b942-505f3ce7b135 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:50.447484 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-4f6b7a23-7601-4e63-b942-505f3ce7b135 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:50.449125 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1310/2639] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:38:50 2022] DELETE /identity/v3/projects/23674d5fe9474fa5a27795efc6dc9f76 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:38:50.454983 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5a12a-1bb2-4450-86b3-ed5aeff2f4ee None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:50.455444 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5a12a-1bb2-4450-86b3-ed5aeff2f4ee None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:50.455585 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5a12a-1bb2-4450-86b3-ed5aeff2f4ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:50.474896 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-3cb5a12a-1bb2-4450-86b3-ed5aeff2f4ee None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:38:50.476095 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-3cb5a12a-1bb2-4450-86b3-ed5aeff2f4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:50.535230 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1330/2640] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:38:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:38:50.543120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66ffd9-04d5-47c5-9bef-43d8efa1fe14 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:50.543441 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66ffd9-04d5-47c5-9bef-43d8efa1fe14 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:50.543745 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66ffd9-04d5-47c5-9bef-43d8efa1fe14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:50.563265 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-ae66ffd9-04d5-47c5-9bef-43d8efa1fe14 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:38:50.564576 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-ae66ffd9-04d5-47c5-9bef-43d8efa1fe14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:50.575092 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1311/2641] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:38:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:38:50.579057 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5a12a-1bb2-4450-86b3-ed5aeff2f4ee None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:50.589268 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5a12a-1bb2-4450-86b3-ed5aeff2f4ee None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:50.594080 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1fad12-7161-4e25-9e43-3a9dbfc230bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:50.594790 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1fad12-7161-4e25-9e43-3a9dbfc230bc admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:50.595813 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1fad12-7161-4e25-9e43-3a9dbfc230bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:50.596128 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1fad12-7161-4e25-9e43-3a9dbfc230bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:50.596847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1fad12-7161-4e25-9e43-3a9dbfc230bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:38:50.597160 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1fad12-7161-4e25-9e43-3a9dbfc230bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:50.598773 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1fad12-7161-4e25-9e43-3a9dbfc230bc admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:50.613873 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1331/2642] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:38:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:38:50.619786 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae66ffd9-04d5-47c5-9bef-43d8efa1fe14 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:50.629519 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae66ffd9-04d5-47c5-9bef-43d8efa1fe14 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:50.634359 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4f0f82-bae4-411f-8cff-39e75db8ecf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:50.634979 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f0f82-bae4-411f-8cff-39e75db8ecf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:50.635267 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f0f82-bae4-411f-8cff-39e75db8ecf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:50.635546 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f0f82-bae4-411f-8cff-39e75db8ecf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:50.636292 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4f0f82-bae4-411f-8cff-39e75db8ecf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:50.638188 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4f0f82-bae4-411f-8cff-39e75db8ecf5 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:50.713583 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1312/2643] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:38:50 2022] POST /identity/v3/projects => generated 371 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:38:50.717657 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1fad12-7161-4e25-9e43-3a9dbfc230bc admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:50.728499 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1fad12-7161-4e25-9e43-3a9dbfc230bc admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:50.785835 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a5c4c2-c5f1-40be-8d36-6937b8556749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:50.786537 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5c4c2-c5f1-40be-8d36-6937b8556749 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:50.786829 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5c4c2-c5f1-40be-8d36-6937b8556749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:50.787107 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5c4c2-c5f1-40be-8d36-6937b8556749 admin admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:50.787834 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a5c4c2-c5f1-40be-8d36-6937b8556749 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:50.789836 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a5c4c2-c5f1-40be-8d36-6937b8556749 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:50.861632 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1332/2644] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:38:50 2022] POST /identity/v3/users => generated 286 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:38:50.865824 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4f0f82-bae4-411f-8cff-39e75db8ecf5 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:50.881653 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4f0f82-bae4-411f-8cff-39e75db8ecf5 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:50.886276 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f350a-dcb4-4de5-8846-adf7948ac26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:50.888456 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f350a-dcb4-4de5-8846-adf7948ac26f admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:50.888456 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f350a-dcb4-4de5-8846-adf7948ac26f admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:50.888456 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f350a-dcb4-4de5-8846-adf7948ac26f admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:50.888456 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9f350a-dcb4-4de5-8846-adf7948ac26f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:50.889757 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9f350a-dcb4-4de5-8846-adf7948ac26f admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:50.901123 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1313/2645] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:38:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:38:50.905064 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a5c4c2-c5f1-40be-8d36-6937b8556749 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:50.915050 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a5c4c2-c5f1-40be-8d36-6937b8556749 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:50.920117 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c16ba-c349-4dd5-ad1b-448c01de14bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:50.920747 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c16ba-c349-4dd5-ad1b-448c01de14bc admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:50.921158 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c16ba-c349-4dd5-ad1b-448c01de14bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:50.921304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c16ba-c349-4dd5-ad1b-448c01de14bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:50.922080 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660c16ba-c349-4dd5-ad1b-448c01de14bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:50.923670 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660c16ba-c349-4dd5-ad1b-448c01de14bc admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:50.934456 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1333/2646] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:38:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:38:50.939315 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f350a-dcb4-4de5-8846-adf7948ac26f admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:50.950527 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f350a-dcb4-4de5-8846-adf7948ac26f admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:50.956104 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055887f6-9ea9-47f5-886d-7993cbc13d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:50.956801 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055887f6-9ea9-47f5-886d-7993cbc13d98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:50.957100 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055887f6-9ea9-47f5-886d-7993cbc13d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:50.957372 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055887f6-9ea9-47f5-886d-7993cbc13d98 admin admin] PATH_INFO: `/v3/projects/7b3f679565d546ff8e6418643699c002/users/f2a7a3b126294571a8586b6e7b43056c/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:50.972663 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055887f6-9ea9-47f5-886d-7993cbc13d98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b3f679565d546ff8e6418643699c002, user_id=f2a7a3b126294571a8586b6e7b43056c, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:50.975227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055887f6-9ea9-47f5-886d-7993cbc13d98 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:51.049114 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1314/2647] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:38:50 2022] PUT /identity/v3/projects/7b3f679565d546ff8e6418643699c002/users/f2a7a3b126294571a8586b6e7b43056c/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:38:51.054142 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697db721-059d-460b-9151-750fa707b796 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:51.054462 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697db721-059d-460b-9151-750fa707b796 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:51.054921 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697db721-059d-460b-9151-750fa707b796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:51.074388 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-697db721-059d-460b-9151-750fa707b796 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:38:51.075597 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-697db721-059d-460b-9151-750fa707b796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:51.084547 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:38:51.135077 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1334/2648] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:38:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:38:51.145759 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055887f6-9ea9-47f5-886d-7993cbc13d98 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:51.147449 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-055887f6-9ea9-47f5-886d-7993cbc13d98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:51.205552 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055887f6-9ea9-47f5-886d-7993cbc13d98 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:51.209494 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df93ec-6e68-4f1e-8491-98b734743a8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:51.210109 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df93ec-6e68-4f1e-8491-98b734743a8c service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:51.210396 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df93ec-6e68-4f1e-8491-98b734743a8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:51.210677 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df93ec-6e68-4f1e-8491-98b734743a8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:51.219623 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88df93ec-6e68-4f1e-8491-98b734743a8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:51.221112 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88df93ec-6e68-4f1e-8491-98b734743a8c service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:51.230552 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1315/2649] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:38:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:38:51.649380 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697db721-059d-460b-9151-750fa707b796 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:51.651234 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-697db721-059d-460b-9151-750fa707b796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:51.663883 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697db721-059d-460b-9151-750fa707b796 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:51.669419 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c2d52c-a8ac-4361-b96f-e45c0079725d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:51.673051 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c2d52c-a8ac-4361-b96f-e45c0079725d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:51.673051 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c2d52c-a8ac-4361-b96f-e45c0079725d admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:51.673051 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c2d52c-a8ac-4361-b96f-e45c0079725d admin admin] PATH_INFO: `/v3/users/22f4a016c52647db95dfdd2c29c7bec3` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:51.673051 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c2d52c-a8ac-4361-b96f-e45c0079725d admin admin] RBAC: Authorizing `identity:delete_user(user_id=22f4a016c52647db95dfdd2c29c7bec3)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:51.673994 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c2d52c-a8ac-4361-b96f-e45c0079725d admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:51.887875 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c8c2d52c-a8ac-4361-b96f-e45c0079725d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22f4a016c52647db95dfdd2c29c7bec3'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:51.888705 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c8c2d52c-a8ac-4361-b96f-e45c0079725d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22f4a016c52647db95dfdd2c29c7bec3'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:51.904193 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c8c2d52c-a8ac-4361-b96f-e45c0079725d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22f4a016c52647db95dfdd2c29c7bec3'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:52.026595 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c8c2d52c-a8ac-4361-b96f-e45c0079725d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22f4a016c52647db95dfdd2c29c7bec3'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:52.034467 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-c8c2d52c-a8ac-4361-b96f-e45c0079725d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22f4a016c52647db95dfdd2c29c7bec3'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:52.041976 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1335/2650] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:38:51 2022] DELETE /identity/v3/users/22f4a016c52647db95dfdd2c29c7bec3 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:38:52.046346 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b8cba0-84fe-45e8-b5d0-15b022cbe542 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:52.046643 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b8cba0-84fe-45e8-b5d0-15b022cbe542 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:52.046946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b8cba0-84fe-45e8-b5d0-15b022cbe542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:52.066812 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-c5b8cba0-84fe-45e8-b5d0-15b022cbe542 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:38:52.068042 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-c5b8cba0-84fe-45e8-b5d0-15b022cbe542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:52.129218 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1316/2651] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:38:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:38:52.139023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c2d52c-a8ac-4361-b96f-e45c0079725d admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:52.140719 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-c8c2d52c-a8ac-4361-b96f-e45c0079725d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:52.204509 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c2d52c-a8ac-4361-b96f-e45c0079725d admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:52.208094 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fed21a-3174-46c4-9d1a-efe99fd0159d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:52.208716 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fed21a-3174-46c4-9d1a-efe99fd0159d service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:52.209465 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fed21a-3174-46c4-9d1a-efe99fd0159d service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:52.209465 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fed21a-3174-46c4-9d1a-efe99fd0159d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:52.219407 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fed21a-3174-46c4-9d1a-efe99fd0159d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:52.220932 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fed21a-3174-46c4-9d1a-efe99fd0159d service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:52.231840 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1336/2652] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:38:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:38:52.474698 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b8cba0-84fe-45e8-b5d0-15b022cbe542 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:52.476143 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-c5b8cba0-84fe-45e8-b5d0-15b022cbe542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:52.492153 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b8cba0-84fe-45e8-b5d0-15b022cbe542 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:52.507762 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac00c0d3-1990-4312-963b-9346f545ed76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:52.507762 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac00c0d3-1990-4312-963b-9346f545ed76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:52.507762 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac00c0d3-1990-4312-963b-9346f545ed76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:52.507762 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac00c0d3-1990-4312-963b-9346f545ed76 admin admin] PATH_INFO: `/v3/projects/acd3963551a44ea8a6f9f5829f7d2332` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:52.508302 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac00c0d3-1990-4312-963b-9346f545ed76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acd3963551a44ea8a6f9f5829f7d2332)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:52.508302 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac00c0d3-1990-4312-963b-9346f545ed76 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:52.513658 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:38:52.621343 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-ac00c0d3-1990-4312-963b-9346f545ed76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acd3963551a44ea8a6f9f5829f7d2332', 'request_id': 'req-ac00c0d3-1990-4312-963b-9346f545ed76'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:52.628501 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-ac00c0d3-1990-4312-963b-9346f545ed76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acd3963551a44ea8a6f9f5829f7d2332', 'request_id': 'req-ac00c0d3-1990-4312-963b-9346f545ed76'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:52.712919 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-ac00c0d3-1990-4312-963b-9346f545ed76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acd3963551a44ea8a6f9f5829f7d2332', 'request_id': 'req-ac00c0d3-1990-4312-963b-9346f545ed76'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:52.725532 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-ac00c0d3-1990-4312-963b-9346f545ed76 admin admin] The token cache is being invalidate because project acd3963551a44ea8a6f9f5829f7d2332 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:38:52.725975 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-ac00c0d3-1990-4312-963b-9346f545ed76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:52.727074 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-ac00c0d3-1990-4312-963b-9346f545ed76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:38:52.746355 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1317/2653] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:38:52 2022] DELETE /identity/v3/projects/acd3963551a44ea8a6f9f5829f7d2332 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:38:52.751619 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d250013-6a2b-41f7-b0e6-fd9a770062a0 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:52.752188 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d250013-6a2b-41f7-b0e6-fd9a770062a0 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:52.752295 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d250013-6a2b-41f7-b0e6-fd9a770062a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:52.773159 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-4d250013-6a2b-41f7-b0e6-fd9a770062a0 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:38:52.774406 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-4d250013-6a2b-41f7-b0e6-fd9a770062a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:52.803012 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:38:52.864290 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1337/2654] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:38:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:38:52.872436 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87701030-843e-4c33-a1de-8317dfa4ce49 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:52.872655 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87701030-843e-4c33-a1de-8317dfa4ce49 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:52.872946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87701030-843e-4c33-a1de-8317dfa4ce49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:52.896083 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-87701030-843e-4c33-a1de-8317dfa4ce49 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:38:52.897342 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-87701030-843e-4c33-a1de-8317dfa4ce49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:52.906819 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1318/2655] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:38:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:38:52.911592 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d250013-6a2b-41f7-b0e6-fd9a770062a0 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:52.922372 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d250013-6a2b-41f7-b0e6-fd9a770062a0 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:52.929311 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32eeb7-8c4a-4673-a668-c5b3d01cd247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:52.930304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32eeb7-8c4a-4673-a668-c5b3d01cd247 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:52.930304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32eeb7-8c4a-4673-a668-c5b3d01cd247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:52.936041 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32eeb7-8c4a-4673-a668-c5b3d01cd247 admin admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:52.936041 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a32eeb7-8c4a-4673-a668-c5b3d01cd247 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:38:52.936041 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a32eeb7-8c4a-4673-a668-c5b3d01cd247 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:52.936041 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a32eeb7-8c4a-4673-a668-c5b3d01cd247 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:52.951065 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1338/2656] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:38:52.955988 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87701030-843e-4c33-a1de-8317dfa4ce49 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:52.966130 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87701030-843e-4c33-a1de-8317dfa4ce49 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:52.972284 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52846062-2351-4000-96ea-f89c74d65184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:52.973236 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52846062-2351-4000-96ea-f89c74d65184 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:52.973943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52846062-2351-4000-96ea-f89c74d65184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:52.974382 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52846062-2351-4000-96ea-f89c74d65184 admin admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:52.975270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52846062-2351-4000-96ea-f89c74d65184 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:52.977520 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52846062-2351-4000-96ea-f89c74d65184 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:53.091184 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1319/2657] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:38:52 2022] POST /identity/v3/projects => generated 383 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:38:53.096160 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32eeb7-8c4a-4673-a668-c5b3d01cd247 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:53.113450 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32eeb7-8c4a-4673-a668-c5b3d01cd247 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:53.192122 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b01d886-94eb-4334-8b30-0ac04479c9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:53.192751 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01d886-94eb-4334-8b30-0ac04479c9bc admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:53.193038 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01d886-94eb-4334-8b30-0ac04479c9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:53.193320 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01d886-94eb-4334-8b30-0ac04479c9bc admin admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:53.193993 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b01d886-94eb-4334-8b30-0ac04479c9bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:53.196023 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b01d886-94eb-4334-8b30-0ac04479c9bc admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:53.337064 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1339/2658] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:38:53 2022] POST /identity/v3/users => generated 292 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:38:53.342447 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52846062-2351-4000-96ea-f89c74d65184 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:53.353385 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52846062-2351-4000-96ea-f89c74d65184 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:53.359737 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346038b2-9620-4d11-91e2-b4656edcb8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:53.360722 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346038b2-9620-4d11-91e2-b4656edcb8c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:53.361363 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346038b2-9620-4d11-91e2-b4656edcb8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:53.361872 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346038b2-9620-4d11-91e2-b4656edcb8c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:53.362791 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346038b2-9620-4d11-91e2-b4656edcb8c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:53.364690 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346038b2-9620-4d11-91e2-b4656edcb8c1 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:53.378474 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1320/2659] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:38:53.382144 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b01d886-94eb-4334-8b30-0ac04479c9bc admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:53.394694 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b01d886-94eb-4334-8b30-0ac04479c9bc admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:53.400863 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14068ec5-3482-43d7-86bf-65346b80e567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:53.401497 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14068ec5-3482-43d7-86bf-65346b80e567 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:53.401787 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14068ec5-3482-43d7-86bf-65346b80e567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:53.402065 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14068ec5-3482-43d7-86bf-65346b80e567 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:53.402904 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14068ec5-3482-43d7-86bf-65346b80e567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:53.404551 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14068ec5-3482-43d7-86bf-65346b80e567 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:53.420898 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1340/2660] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:38:53.423658 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346038b2-9620-4d11-91e2-b4656edcb8c1 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:53.436821 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346038b2-9620-4d11-91e2-b4656edcb8c1 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:53.444650 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb79f0-02ad-4cdd-9b3f-9ab10979700d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:53.445521 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb79f0-02ad-4cdd-9b3f-9ab10979700d admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:53.446924 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb79f0-02ad-4cdd-9b3f-9ab10979700d admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:53.447599 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb79f0-02ad-4cdd-9b3f-9ab10979700d admin admin] PATH_INFO: `/v3/projects/39c62721dfdd46e3978fccce91b63791/users/8cfb345e7c044b669ec6c4a6a79a28ee/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:53.464225 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cb79f0-02ad-4cdd-9b3f-9ab10979700d admin admin] RBAC: Authorizing `identity:create_grant(project_id=39c62721dfdd46e3978fccce91b63791, user_id=8cfb345e7c044b669ec6c4a6a79a28ee, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:53.467063 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cb79f0-02ad-4cdd-9b3f-9ab10979700d admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:53.557177 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1321/2661] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:38:53 2022] PUT /identity/v3/projects/39c62721dfdd46e3978fccce91b63791/users/8cfb345e7c044b669ec6c4a6a79a28ee/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:38:53.562056 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e0160c-710c-4800-835b-1353e1939d1a None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:53.563489 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e0160c-710c-4800-835b-1353e1939d1a None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:53.563489 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e0160c-710c-4800-835b-1353e1939d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:53.586229 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-d8e0160c-710c-4800-835b-1353e1939d1a None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:38:53.587455 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-d8e0160c-710c-4800-835b-1353e1939d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:53.681581 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1341/2662] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:38:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:38:53.696773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb79f0-02ad-4cdd-9b3f-9ab10979700d admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:53.698646 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-a5cb79f0-02ad-4cdd-9b3f-9ab10979700d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:53.810991 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb79f0-02ad-4cdd-9b3f-9ab10979700d admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:53.816169 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b760a565-2d2b-4695-abe2-333fc2270609 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:53.816868 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760a565-2d2b-4695-abe2-333fc2270609 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:53.817160 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760a565-2d2b-4695-abe2-333fc2270609 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:53.817438 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760a565-2d2b-4695-abe2-333fc2270609 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:53.827804 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b760a565-2d2b-4695-abe2-333fc2270609 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:53.829360 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b760a565-2d2b-4695-abe2-333fc2270609 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:53.908986 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:38:53.917437 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1322/2663] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:38:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:38:56.036925 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33891f83-1c28-4275-bcec-7544267aaf09 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:56.037453 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33891f83-1c28-4275-bcec-7544267aaf09 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:56.037453 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33891f83-1c28-4275-bcec-7544267aaf09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:56.090610 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-33891f83-1c28-4275-bcec-7544267aaf09 None None] MFA Rules not processed for user `8cfb345e7c044b669ec6c4a6a79a28ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:38:56.092512 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-33891f83-1c28-4275-bcec-7544267aaf09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:56.109735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:38:56.187881 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1342/2664] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:38:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:38:56.199552 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b760a565-2d2b-4695-abe2-333fc2270609 service neutron] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:56.201150 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-b760a565-2d2b-4695-abe2-333fc2270609 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:56.305967 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b760a565-2d2b-4695-abe2-333fc2270609 service neutron] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:56.310408 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757e024-b396-490b-8bc2-5a32d97f0a5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:56.311045 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757e024-b396-490b-8bc2-5a32d97f0a5d service cinder] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:56.311436 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757e024-b396-490b-8bc2-5a32d97f0a5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:56.311620 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757e024-b396-490b-8bc2-5a32d97f0a5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:56.321215 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b757e024-b396-490b-8bc2-5a32d97f0a5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:56.325759 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b757e024-b396-490b-8bc2-5a32d97f0a5d service cinder] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:56.333300 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1323/2665] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:38:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:38:58.092102 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4e090-2fd2-4ebf-9839-e91e70c6e873 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:58.095948 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4e090-2fd2-4ebf-9839-e91e70c6e873 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:58.095948 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4e090-2fd2-4ebf-9839-e91e70c6e873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:58.128677 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-34d4e090-2fd2-4ebf-9839-e91e70c6e873 None None] MFA Rules not processed for user `f2a7a3b126294571a8586b6e7b43056c`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:38:58.134989 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-34d4e090-2fd2-4ebf-9839-e91e70c6e873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:58.217753 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1343/2666] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:38:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:38:58.222179 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757e024-b396-490b-8bc2-5a32d97f0a5d service cinder] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:58.224579 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-b757e024-b396-490b-8bc2-5a32d97f0a5d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:58.255857 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757e024-b396-490b-8bc2-5a32d97f0a5d service cinder] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:58.260945 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eba2b0-5554-40a1-930e-8f220bff2ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:58.261585 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eba2b0-5554-40a1-930e-8f220bff2ec7 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:58.261870 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eba2b0-5554-40a1-930e-8f220bff2ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:58.262144 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eba2b0-5554-40a1-930e-8f220bff2ec7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:58.262843 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eba2b0-5554-40a1-930e-8f220bff2ec7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:58.264838 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eba2b0-5554-40a1-930e-8f220bff2ec7 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:58.342757 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1324/2667] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:38:58 2022] POST /identity/v3/projects => generated 371 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:38:58.348175 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d4e090-2fd2-4ebf-9839-e91e70c6e873 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:58.359683 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d4e090-2fd2-4ebf-9839-e91e70c6e873 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:58.423183 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e102323-111a-4a49-899e-8d8388d2b65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:58.424422 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e102323-111a-4a49-899e-8d8388d2b65f admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:58.424642 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e102323-111a-4a49-899e-8d8388d2b65f admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:58.425138 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e102323-111a-4a49-899e-8d8388d2b65f admin admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:58.425832 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e102323-111a-4a49-899e-8d8388d2b65f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:58.429641 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e102323-111a-4a49-899e-8d8388d2b65f admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:58.477331 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1d946-e2fe-4eda-a20a-578dad0d0c71 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:58.477527 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1d946-e2fe-4eda-a20a-578dad0d0c71 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:58.477805 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1d946-e2fe-4eda-a20a-578dad0d0c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:58.501767 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-7df1d946-e2fe-4eda-a20a-578dad0d0c71 None None] MFA Rules not processed for user `8cfb345e7c044b669ec6c4a6a79a28ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:38:58.503085 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-7df1d946-e2fe-4eda-a20a-578dad0d0c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:38:58.558083 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1344/2668] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:38:58 2022] POST /identity/v3/users => generated 292 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:38:58.559152 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e102323-111a-4a49-899e-8d8388d2b65f admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:58.569742 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1325/2669] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:38:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:38:58.570693 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e102323-111a-4a49-899e-8d8388d2b65f admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:58.575740 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3199121-2129-4601-b3d5-d5f3a8fba01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:58.576407 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3199121-2129-4601-b3d5-d5f3a8fba01b admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:58.576688 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3199121-2129-4601-b3d5-d5f3a8fba01b admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:58.576963 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3199121-2129-4601-b3d5-d5f3a8fba01b admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:58.578129 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3199121-2129-4601-b3d5-d5f3a8fba01b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:58.579300 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3199121-2129-4601-b3d5-d5f3a8fba01b admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:58.591370 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1345/2670] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:38:58.596053 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df1d946-e2fe-4eda-a20a-578dad0d0c71 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:58.608331 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df1d946-e2fe-4eda-a20a-578dad0d0c71 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:58.614427 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2196e83e-0e17-4299-93e1-56fb7fd28d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:58.615108 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196e83e-0e17-4299-93e1-56fb7fd28d14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:58.615390 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196e83e-0e17-4299-93e1-56fb7fd28d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:58.615673 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196e83e-0e17-4299-93e1-56fb7fd28d14 admin admin] PATH_INFO: `/v3/projects/5bf684650c2145cbb6d131625235ce7e/users/d14cd2362c3b4ccebf8403728b87d020/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:58.636353 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2196e83e-0e17-4299-93e1-56fb7fd28d14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bf684650c2145cbb6d131625235ce7e, user_id=d14cd2362c3b4ccebf8403728b87d020, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:58.639485 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2196e83e-0e17-4299-93e1-56fb7fd28d14 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:58.747588 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1326/2671] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:38:58 2022] PUT /identity/v3/projects/5bf684650c2145cbb6d131625235ce7e/users/d14cd2362c3b4ccebf8403728b87d020/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:38:58.755836 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3199121-2129-4601-b3d5-d5f3a8fba01b admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:58.762169 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3199121-2129-4601-b3d5-d5f3a8fba01b admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:58.786259 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:38:58.831147 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f152c-1a40-4686-b28e-936ffe490f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:58.832050 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f152c-1a40-4686-b28e-936ffe490f66 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:58.832773 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f152c-1a40-4686-b28e-936ffe490f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:58.833061 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f152c-1a40-4686-b28e-936ffe490f66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:58.833867 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180f152c-1a40-4686-b28e-936ffe490f66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:58.835569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180f152c-1a40-4686-b28e-936ffe490f66 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:58.847269 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1346/2672] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:38:58.852932 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2196e83e-0e17-4299-93e1-56fb7fd28d14 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:38:58.876451 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2196e83e-0e17-4299-93e1-56fb7fd28d14 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:38:58.883995 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdcd2ca-f43c-47f9-9f6d-f1d350de4367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:38:58.885049 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdcd2ca-f43c-47f9-9f6d-f1d350de4367 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:38:58.885481 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdcd2ca-f43c-47f9-9f6d-f1d350de4367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:38:58.885912 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdcd2ca-f43c-47f9-9f6d-f1d350de4367 admin admin] PATH_INFO: `/v3/projects/5bf684650c2145cbb6d131625235ce7e/users/d14cd2362c3b4ccebf8403728b87d020/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:38:58.888801 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdcd2ca-f43c-47f9-9f6d-f1d350de4367 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bf684650c2145cbb6d131625235ce7e, user_id=d14cd2362c3b4ccebf8403728b87d020, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:38:58.892498 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdcd2ca-f43c-47f9-9f6d-f1d350de4367 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:38:58.905101 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1327/2673] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:38:58 2022] PUT /identity/v3/projects/5bf684650c2145cbb6d131625235ce7e/users/d14cd2362c3b4ccebf8403728b87d020/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:39:05.580520 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33fdcc-e105-4681-9eea-5f670a765c39 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:39:05.580993 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33fdcc-e105-4681-9eea-5f670a765c39 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:39:05.581078 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33fdcc-e105-4681-9eea-5f670a765c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:39:05.618083 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-1a33fdcc-e105-4681-9eea-5f670a765c39 None None] MFA Rules not processed for user `d14cd2362c3b4ccebf8403728b87d020`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:39:05.619414 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-1a33fdcc-e105-4681-9eea-5f670a765c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:39:05.702020 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1347/2674] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:39:05 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:39:05.717163 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdcd2ca-f43c-47f9-9f6d-f1d350de4367 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:39:05.718890 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-9cdcd2ca-f43c-47f9-9f6d-f1d350de4367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:39:05.744541 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:39:05.837782 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdcd2ca-f43c-47f9-9f6d-f1d350de4367 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:39:05.843762 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db44213-8466-412e-a47d-bdd34bdfb81a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:39:05.847525 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db44213-8466-412e-a47d-bdd34bdfb81a service nova] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:39:05.847525 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db44213-8466-412e-a47d-bdd34bdfb81a service nova] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:39:05.847525 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db44213-8466-412e-a47d-bdd34bdfb81a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:39:05.906665 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db44213-8466-412e-a47d-bdd34bdfb81a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:39:05.908960 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db44213-8466-412e-a47d-bdd34bdfb81a service nova] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:39:05.924262 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1328/2675] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:39:05 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:39:08.183392 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33fdcc-e105-4681-9eea-5f670a765c39 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:39:08.184846 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-1a33fdcc-e105-4681-9eea-5f670a765c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:39:08.285829 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33fdcc-e105-4681-9eea-5f670a765c39 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:39:08.294947 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e652cd-d911-4b13-8290-56dba31295f9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '107a7c18a2c6440b86e1b7c3e39cc724', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:39:08.295663 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e652cd-d911-4b13-8290-56dba31295f9 service placement] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:39:08.295952 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e652cd-d911-4b13-8290-56dba31295f9 service placement] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:39:08.296472 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e652cd-d911-4b13-8290-56dba31295f9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:39:08.298306 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-26e652cd-d911-4b13-8290-56dba31295f9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:39:08.310981 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e652cd-d911-4b13-8290-56dba31295f9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:39:08.312711 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e652cd-d911-4b13-8290-56dba31295f9 service placement] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:39:08.324101 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1348/2676] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:39:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:39:08.685025 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02a560-e645-4ab3-92e8-f2fe30683ecd None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:39:08.685585 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02a560-e645-4ab3-92e8-f2fe30683ecd None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:39:08.685585 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02a560-e645-4ab3-92e8-f2fe30683ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:39:08.705435 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-ae02a560-e645-4ab3-92e8-f2fe30683ecd None None] MFA Rules not processed for user `f2a7a3b126294571a8586b6e7b43056c`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:39:08.706635 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-ae02a560-e645-4ab3-92e8-f2fe30683ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:39:08.713672 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1329/2677] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:39:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:39:13.440085 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eba8c1-e3c7-4997-81ab-bd4a83894729 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:39:13.440536 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eba8c1-e3c7-4997-81ab-bd4a83894729 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:39:13.440678 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eba8c1-e3c7-4997-81ab-bd4a83894729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:39:13.460285 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-39eba8c1-e3c7-4997-81ab-bd4a83894729 None None] MFA Rules not processed for user `dd5c43d63a754eeaaba429f5c8153bba`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:39:13.461502 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-39eba8c1-e3c7-4997-81ab-bd4a83894729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:39:13.471151 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:39:13.517940 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1349/2678] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:39:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:39:18.699755 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=63 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:39:23.510332 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=18 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:39:53.677734 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02a560-e645-4ab3-92e8-f2fe30683ecd None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:39:53.679741 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-ae02a560-e645-4ab3-92e8-f2fe30683ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:39:53.755211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02a560-e645-4ab3-92e8-f2fe30683ecd None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:39:53.761693 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe0812-f305-4ff3-a6ad-70f887000ff7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:39:53.762031 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe0812-f305-4ff3-a6ad-70f887000ff7 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:39:53.762318 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe0812-f305-4ff3-a6ad-70f887000ff7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:39:53.762596 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe0812-f305-4ff3-a6ad-70f887000ff7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:39:53.776932 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fe0812-f305-4ff3-a6ad-70f887000ff7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:39:53.778678 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fe0812-f305-4ff3-a6ad-70f887000ff7 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:39:53.789130 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1330/2679] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:39:53 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:40:03.784096 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=35 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:40:15.978687 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574fc0c-f530-4c46-891c-bc5ffa7ea2db None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:40:15.979197 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574fc0c-f530-4c46-891c-bc5ffa7ea2db None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:40:15.979304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574fc0c-f530-4c46-891c-bc5ffa7ea2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:40:15.998258 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-e574fc0c-f530-4c46-891c-bc5ffa7ea2db None None] MFA Rules not processed for user `8cfb345e7c044b669ec6c4a6a79a28ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:40:15.999474 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-e574fc0c-f530-4c46-891c-bc5ffa7ea2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:40:16.055698 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1350/2680] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:40:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:40:26.048318 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=25 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:40:33.201212 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32bf73f-0056-4750-8f88-ef6d938b7589 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:40:33.201784 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32bf73f-0056-4750-8f88-ef6d938b7589 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:40:33.201784 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32bf73f-0056-4750-8f88-ef6d938b7589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:40:33.220705 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-c32bf73f-0056-4750-8f88-ef6d938b7589 None None] MFA Rules not processed for user `dd5c43d63a754eeaaba429f5c8153bba`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:40:33.221929 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-c32bf73f-0056-4750-8f88-ef6d938b7589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:40:33.229236 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1331/2681] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:40:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:40:43.215796 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=3 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:40:48.832055 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574fc0c-f530-4c46-891c-bc5ffa7ea2db None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:40:48.842259 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574fc0c-f530-4c46-891c-bc5ffa7ea2db None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:40:48.845889 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033b773-1e47-4793-a1bf-a8f81c8681ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:40:48.846496 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033b773-1e47-4793-a1bf-a8f81c8681ca service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:40:48.846802 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033b773-1e47-4793-a1bf-a8f81c8681ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:40:48.847077 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033b773-1e47-4793-a1bf-a8f81c8681ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:40:48.848833 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-9033b773-1e47-4793-a1bf-a8f81c8681ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:40:48.863134 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9033b773-1e47-4793-a1bf-a8f81c8681ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:40:48.864651 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9033b773-1e47-4793-a1bf-a8f81c8681ca service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:40:48.876122 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1351/2682] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:40:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:40:49.794427 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32bf73f-0056-4750-8f88-ef6d938b7589 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:40:49.804109 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32bf73f-0056-4750-8f88-ef6d938b7589 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:40:49.808188 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee7fdfb-6784-4d6f-aec7-b8c7c3c3ca2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:40:49.808789 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7fdfb-6784-4d6f-aec7-b8c7c3c3ca2c service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:40:49.809067 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7fdfb-6784-4d6f-aec7-b8c7c3c3ca2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:40:49.809428 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7fdfb-6784-4d6f-aec7-b8c7c3c3ca2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:40:49.811434 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-aee7fdfb-6784-4d6f-aec7-b8c7c3c3ca2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:40:49.826077 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee7fdfb-6784-4d6f-aec7-b8c7c3c3ca2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:40:49.827455 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee7fdfb-6784-4d6f-aec7-b8c7c3c3ca2c service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:40:49.838417 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1332/2683] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:40:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:40:55.760329 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033b773-1e47-4793-a1bf-a8f81c8681ca service neutron] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:40:55.769276 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033b773-1e47-4793-a1bf-a8f81c8681ca service neutron] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:40:55.773061 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe4727-99e1-4153-86f2-ca193e46a689 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:40:55.779955 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe4727-99e1-4153-86f2-ca193e46a689 service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:40:55.779955 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe4727-99e1-4153-86f2-ca193e46a689 service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:40:55.779955 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe4727-99e1-4153-86f2-ca193e46a689 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:40:55.779955 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-a7fe4727-99e1-4153-86f2-ca193e46a689 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:40:55.842616 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fe4727-99e1-4153-86f2-ca193e46a689 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:40:55.844151 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fe4727-99e1-4153-86f2-ca193e46a689 service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:40:55.854418 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1352/2684] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:40:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:40:59.833590 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=6 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:41:03.789555 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee7fdfb-6784-4d6f-aec7-b8c7c3c3ca2c service neutron] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:41:03.791733 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-aee7fdfb-6784-4d6f-aec7-b8c7c3c3ca2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:41:03.806612 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee7fdfb-6784-4d6f-aec7-b8c7c3c3ca2c service neutron] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:41:03.811202 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b0b9fd-9a02-4fcd-9c7d-feb17eb02340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:41:03.811831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b0b9fd-9a02-4fcd-9c7d-feb17eb02340 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:41:03.812148 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b0b9fd-9a02-4fcd-9c7d-feb17eb02340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:41:03.812426 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b0b9fd-9a02-4fcd-9c7d-feb17eb02340 admin admin] PATH_INFO: `/v3/users/1782b4e84c9f484297e17c248738105c` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:41:03.813608 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b0b9fd-9a02-4fcd-9c7d-feb17eb02340 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1782b4e84c9f484297e17c248738105c)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:41:03.815816 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b0b9fd-9a02-4fcd-9c7d-feb17eb02340 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:41:03.994571 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-e8b0b9fd-9a02-4fcd-9c7d-feb17eb02340 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1782b4e84c9f484297e17c248738105c'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:41:04.001553 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-e8b0b9fd-9a02-4fcd-9c7d-feb17eb02340 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1782b4e84c9f484297e17c248738105c'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:41:04.002246 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-e8b0b9fd-9a02-4fcd-9c7d-feb17eb02340 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1782b4e84c9f484297e17c248738105c'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:41:04.013880 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-e8b0b9fd-9a02-4fcd-9c7d-feb17eb02340 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1782b4e84c9f484297e17c248738105c'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:41:04.014510 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-e8b0b9fd-9a02-4fcd-9c7d-feb17eb02340 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1782b4e84c9f484297e17c248738105c'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:41:04.111688 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1333/2685] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:41:03 2022] DELETE /identity/v3/users/1782b4e84c9f484297e17c248738105c => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:41:04.384136 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe4727-99e1-4153-86f2-ca193e46a689 service neutron] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:41:04.394711 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe4727-99e1-4153-86f2-ca193e46a689 service neutron] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:41:04.452284 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5176350-b627-4f5d-82ae-d2ae7cb0e91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:41:04.452928 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5176350-b627-4f5d-82ae-d2ae7cb0e91c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:41:04.453210 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5176350-b627-4f5d-82ae-d2ae7cb0e91c admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:41:04.453487 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5176350-b627-4f5d-82ae-d2ae7cb0e91c admin admin] PATH_INFO: `/v3/projects/99b26247f85d45c0a7c1e763758f2e69` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:41:04.454639 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5176350-b627-4f5d-82ae-d2ae7cb0e91c admin admin] RBAC: Authorizing `identity:delete_project(project_id=99b26247f85d45c0a7c1e763758f2e69)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:41:04.456587 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5176350-b627-4f5d-82ae-d2ae7cb0e91c admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:41:04.565295 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-f5176350-b627-4f5d-82ae-d2ae7cb0e91c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99b26247f85d45c0a7c1e763758f2e69', 'request_id': 'req-f5176350-b627-4f5d-82ae-d2ae7cb0e91c'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:41:04.572320 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-f5176350-b627-4f5d-82ae-d2ae7cb0e91c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99b26247f85d45c0a7c1e763758f2e69', 'request_id': 'req-f5176350-b627-4f5d-82ae-d2ae7cb0e91c'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:41:04.578142 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-f5176350-b627-4f5d-82ae-d2ae7cb0e91c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99b26247f85d45c0a7c1e763758f2e69', 'request_id': 'req-f5176350-b627-4f5d-82ae-d2ae7cb0e91c'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:41:04.649873 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-f5176350-b627-4f5d-82ae-d2ae7cb0e91c admin admin] The token cache is being invalidate because project 99b26247f85d45c0a7c1e763758f2e69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:41:04.650082 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-f5176350-b627-4f5d-82ae-d2ae7cb0e91c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:41:04.650704 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-f5176350-b627-4f5d-82ae-d2ae7cb0e91c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:41:04.652243 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1353/2686] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:41:04 2022] DELETE /identity/v3/projects/99b26247f85d45c0a7c1e763758f2e69 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:41:04.656516 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b0b9fd-9a02-4fcd-9c7d-feb17eb02340 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:41:04.657979 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-e8b0b9fd-9a02-4fcd-9c7d-feb17eb02340 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:41:04.671164 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b0b9fd-9a02-4fcd-9c7d-feb17eb02340 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:41:04.727678 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c2a4cc-90d4-44e3-a417-a7890c0096ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:41:04.728349 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2a4cc-90d4-44e3-a417-a7890c0096ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:41:04.728634 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2a4cc-90d4-44e3-a417-a7890c0096ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:41:04.728911 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2a4cc-90d4-44e3-a417-a7890c0096ed admin admin] PATH_INFO: `/v3/users/d62b8944fe534f10b71e305315a75f0f` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:41:04.730083 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c2a4cc-90d4-44e3-a417-a7890c0096ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=d62b8944fe534f10b71e305315a75f0f)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:41:04.731999 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c2a4cc-90d4-44e3-a417-a7890c0096ed admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:41:04.927537 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-f1c2a4cc-90d4-44e3-a417-a7890c0096ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd62b8944fe534f10b71e305315a75f0f'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:41:04.935240 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-f1c2a4cc-90d4-44e3-a417-a7890c0096ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd62b8944fe534f10b71e305315a75f0f'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:41:04.937126 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-f1c2a4cc-90d4-44e3-a417-a7890c0096ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd62b8944fe534f10b71e305315a75f0f'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:41:04.947824 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-f1c2a4cc-90d4-44e3-a417-a7890c0096ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd62b8944fe534f10b71e305315a75f0f'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:41:04.948540 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-f1c2a4cc-90d4-44e3-a417-a7890c0096ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd62b8944fe534f10b71e305315a75f0f'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:41:04.950149 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:41:05.050204 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1334/2687] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:41:04 2022] DELETE /identity/v3/users/d62b8944fe534f10b71e305315a75f0f => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:41:05.293786 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5176350-b627-4f5d-82ae-d2ae7cb0e91c admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:41:05.295270 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-f5176350-b627-4f5d-82ae-d2ae7cb0e91c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:41:05.309067 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5176350-b627-4f5d-82ae-d2ae7cb0e91c admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:41:05.317666 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:41:05.367304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ce147-a394-4cf6-a568-ccf9a7c29c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:41:05.367942 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ce147-a394-4cf6-a568-ccf9a7c29c54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:41:05.368265 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ce147-a394-4cf6-a568-ccf9a7c29c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:41:05.368547 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ce147-a394-4cf6-a568-ccf9a7c29c54 admin admin] PATH_INFO: `/v3/projects/b7974d0a036b4a8ba4478ba49790bf42` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:41:05.369742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ce147-a394-4cf6-a568-ccf9a7c29c54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7974d0a036b4a8ba4478ba49790bf42)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:41:05.371649 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ce147-a394-4cf6-a568-ccf9a7c29c54 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:41:05.495639 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-418ce147-a394-4cf6-a568-ccf9a7c29c54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7974d0a036b4a8ba4478ba49790bf42', 'request_id': 'req-418ce147-a394-4cf6-a568-ccf9a7c29c54'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:41:05.502015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-418ce147-a394-4cf6-a568-ccf9a7c29c54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7974d0a036b4a8ba4478ba49790bf42', 'request_id': 'req-418ce147-a394-4cf6-a568-ccf9a7c29c54'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:41:05.507793 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-418ce147-a394-4cf6-a568-ccf9a7c29c54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7974d0a036b4a8ba4478ba49790bf42', 'request_id': 'req-418ce147-a394-4cf6-a568-ccf9a7c29c54'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:41:05.575538 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-418ce147-a394-4cf6-a568-ccf9a7c29c54 admin admin] The token cache is being invalidate because project b7974d0a036b4a8ba4478ba49790bf42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:41:05.575758 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-418ce147-a394-4cf6-a568-ccf9a7c29c54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:41:05.576419 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-418ce147-a394-4cf6-a568-ccf9a7c29c54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:41:05.577923 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1354/2688] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:41:05 2022] DELETE /identity/v3/projects/b7974d0a036b4a8ba4478ba49790bf42 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:41:15.090849 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:41:15.572302 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:41:25.998352 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c2a4cc-90d4-44e3-a417-a7890c0096ed admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:41:25.999837 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-f1c2a4cc-90d4-44e3-a417-a7890c0096ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:41:26.014828 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c2a4cc-90d4-44e3-a417-a7890c0096ed admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:41:26.070708 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cdebbb-1d9d-4197-b6a4-79c99795f534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:41:26.071340 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cdebbb-1d9d-4197-b6a4-79c99795f534 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:41:26.071622 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cdebbb-1d9d-4197-b6a4-79c99795f534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:41:26.071925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cdebbb-1d9d-4197-b6a4-79c99795f534 admin admin] PATH_INFO: `/v3/users/8cfb345e7c044b669ec6c4a6a79a28ee` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:41:26.073125 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cdebbb-1d9d-4197-b6a4-79c99795f534 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cfb345e7c044b669ec6c4a6a79a28ee)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:41:26.075022 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cdebbb-1d9d-4197-b6a4-79c99795f534 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:41:26.201446 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-64cdebbb-1d9d-4197-b6a4-79c99795f534 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cfb345e7c044b669ec6c4a6a79a28ee'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:41:26.208194 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-64cdebbb-1d9d-4197-b6a4-79c99795f534 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cfb345e7c044b669ec6c4a6a79a28ee'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:41:26.208859 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-64cdebbb-1d9d-4197-b6a4-79c99795f534 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cfb345e7c044b669ec6c4a6a79a28ee'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:41:26.220980 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-64cdebbb-1d9d-4197-b6a4-79c99795f534 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cfb345e7c044b669ec6c4a6a79a28ee'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:41:26.221605 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-64cdebbb-1d9d-4197-b6a4-79c99795f534 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cfb345e7c044b669ec6c4a6a79a28ee'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:41:26.288568 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1335/2689] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:41:25 2022] DELETE /identity/v3/users/8cfb345e7c044b669ec6c4a6a79a28ee => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:41:26.571109 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ce147-a394-4cf6-a568-ccf9a7c29c54 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:41:26.573214 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-418ce147-a394-4cf6-a568-ccf9a7c29c54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:41:26.585791 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ce147-a394-4cf6-a568-ccf9a7c29c54 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:41:26.643659 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca88bf9-1767-4735-b1ec-2a67a72b013d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:41:26.644395 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca88bf9-1767-4735-b1ec-2a67a72b013d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:41:26.644679 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca88bf9-1767-4735-b1ec-2a67a72b013d admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:41:26.644973 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca88bf9-1767-4735-b1ec-2a67a72b013d admin admin] PATH_INFO: `/v3/projects/39c62721dfdd46e3978fccce91b63791` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:41:26.646133 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca88bf9-1767-4735-b1ec-2a67a72b013d admin admin] RBAC: Authorizing `identity:delete_project(project_id=39c62721dfdd46e3978fccce91b63791)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:41:26.648082 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca88bf9-1767-4735-b1ec-2a67a72b013d admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:41:26.765328 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-8ca88bf9-1767-4735-b1ec-2a67a72b013d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39c62721dfdd46e3978fccce91b63791', 'request_id': 'req-8ca88bf9-1767-4735-b1ec-2a67a72b013d'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:41:26.771541 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-8ca88bf9-1767-4735-b1ec-2a67a72b013d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39c62721dfdd46e3978fccce91b63791', 'request_id': 'req-8ca88bf9-1767-4735-b1ec-2a67a72b013d'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:41:26.777726 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-8ca88bf9-1767-4735-b1ec-2a67a72b013d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39c62721dfdd46e3978fccce91b63791', 'request_id': 'req-8ca88bf9-1767-4735-b1ec-2a67a72b013d'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:41:26.827138 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-8ca88bf9-1767-4735-b1ec-2a67a72b013d admin admin] The token cache is being invalidate because project 39c62721dfdd46e3978fccce91b63791 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:41:26.827360 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-8ca88bf9-1767-4735-b1ec-2a67a72b013d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:41:26.827989 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-8ca88bf9-1767-4735-b1ec-2a67a72b013d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:41:26.829521 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1355/2690] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:41:26 2022] DELETE /identity/v3/projects/39c62721dfdd46e3978fccce91b63791 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:41:26.834479 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd6d009-7761-4a05-9fd3-1747a871fcbc None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:41:26.834768 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd6d009-7761-4a05-9fd3-1747a871fcbc None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:41:26.835072 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd6d009-7761-4a05-9fd3-1747a871fcbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:41:26.891317 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-4bd6d009-7761-4a05-9fd3-1747a871fcbc None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:41:26.892571 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-4bd6d009-7761-4a05-9fd3-1747a871fcbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:41:26.937464 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:41:26.952674 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1336/2691] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:41:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:41:26.961538 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c867c10-62ce-4f61-a8a6-c8c25423c928 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:41:26.961838 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c867c10-62ce-4f61-a8a6-c8c25423c928 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:41:26.962110 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c867c10-62ce-4f61-a8a6-c8c25423c928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:41:26.981202 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-6c867c10-62ce-4f61-a8a6-c8c25423c928 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:41:26.983563 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-6c867c10-62ce-4f61-a8a6-c8c25423c928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:41:26.990581 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1356/2692] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:41:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:41:26.995120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd6d009-7761-4a05-9fd3-1747a871fcbc None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:41:27.004491 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd6d009-7761-4a05-9fd3-1747a871fcbc None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:41:27.009044 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a942ba-3cf5-46e2-a082-07c2a50d4bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:41:27.010216 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a942ba-3cf5-46e2-a082-07c2a50d4bfa admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:41:27.010712 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a942ba-3cf5-46e2-a082-07c2a50d4bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:41:27.011227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a942ba-3cf5-46e2-a082-07c2a50d4bfa admin admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:41:27.012224 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a942ba-3cf5-46e2-a082-07c2a50d4bfa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:41:27.012547 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a942ba-3cf5-46e2-a082-07c2a50d4bfa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:41:27.014141 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a942ba-3cf5-46e2-a082-07c2a50d4bfa admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:41:27.025726 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1337/2693] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:41:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:41:27.030031 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c867c10-62ce-4f61-a8a6-c8c25423c928 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:41:27.039307 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c867c10-62ce-4f61-a8a6-c8c25423c928 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:41:27.043959 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484892d5-2c0b-49cf-a3ad-a34e22f0e3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:41:27.044585 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484892d5-2c0b-49cf-a3ad-a34e22f0e3c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:41:27.044868 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484892d5-2c0b-49cf-a3ad-a34e22f0e3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:41:27.045140 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484892d5-2c0b-49cf-a3ad-a34e22f0e3c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:41:27.045821 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484892d5-2c0b-49cf-a3ad-a34e22f0e3c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:41:27.047697 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484892d5-2c0b-49cf-a3ad-a34e22f0e3c6 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:41:27.142667 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1357/2694] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:41:27 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:41:27.146876 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a942ba-3cf5-46e2-a082-07c2a50d4bfa admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:41:27.157123 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a942ba-3cf5-46e2-a082-07c2a50d4bfa admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:41:27.213608 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93355e9d-601f-4620-8a7f-7a0daaba31fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:41:27.214227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93355e9d-601f-4620-8a7f-7a0daaba31fc admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:41:27.214512 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93355e9d-601f-4620-8a7f-7a0daaba31fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:41:27.214791 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93355e9d-601f-4620-8a7f-7a0daaba31fc admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:41:27.215471 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93355e9d-601f-4620-8a7f-7a0daaba31fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:41:27.217418 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93355e9d-601f-4620-8a7f-7a0daaba31fc admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:41:27.308409 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1338/2695] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:41:27 2022] POST /identity/v3/users => generated 293 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:41:27.312536 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484892d5-2c0b-49cf-a3ad-a34e22f0e3c6 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:41:27.323354 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484892d5-2c0b-49cf-a3ad-a34e22f0e3c6 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:41:27.328027 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d4d98-4c9f-4222-9006-77593e8bf3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:41:27.328667 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d4d98-4c9f-4222-9006-77593e8bf3f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:41:27.328969 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d4d98-4c9f-4222-9006-77593e8bf3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:41:27.329237 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d4d98-4c9f-4222-9006-77593e8bf3f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:41:27.329989 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d4d98-4c9f-4222-9006-77593e8bf3f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:41:27.331582 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d4d98-4c9f-4222-9006-77593e8bf3f1 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:41:27.342092 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1358/2696] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:41:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:41:27.345978 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93355e9d-601f-4620-8a7f-7a0daaba31fc admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:41:27.355992 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93355e9d-601f-4620-8a7f-7a0daaba31fc admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:41:27.360716 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2f6e0-dae8-4c75-9fe4-1dafa17c7ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:41:27.361306 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2f6e0-dae8-4c75-9fe4-1dafa17c7ee4 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:41:27.361587 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2f6e0-dae8-4c75-9fe4-1dafa17c7ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:41:27.361863 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2f6e0-dae8-4c75-9fe4-1dafa17c7ee4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:41:27.362550 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d2f6e0-dae8-4c75-9fe4-1dafa17c7ee4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:41:27.364221 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d2f6e0-dae8-4c75-9fe4-1dafa17c7ee4 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:41:27.377745 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1339/2697] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:41:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:41:27.378860 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d4d98-4c9f-4222-9006-77593e8bf3f1 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:41:27.389500 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d4d98-4c9f-4222-9006-77593e8bf3f1 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:41:27.396434 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da39c38-0bfa-4e1a-bbcd-3be6fe980ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:41:27.396434 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da39c38-0bfa-4e1a-bbcd-3be6fe980ec2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:41:27.396434 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da39c38-0bfa-4e1a-bbcd-3be6fe980ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:41:27.396434 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da39c38-0bfa-4e1a-bbcd-3be6fe980ec2 admin admin] PATH_INFO: `/v3/projects/6e62e9372a7a47528e7973537bebbfb6/users/8ffd179bd4a445eaafb53d7accede863/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:41:27.411217 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da39c38-0bfa-4e1a-bbcd-3be6fe980ec2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e62e9372a7a47528e7973537bebbfb6, user_id=8ffd179bd4a445eaafb53d7accede863, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:41:27.413907 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da39c38-0bfa-4e1a-bbcd-3be6fe980ec2 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:41:27.522802 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1359/2698] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:41:27 2022] PUT /identity/v3/projects/6e62e9372a7a47528e7973537bebbfb6/users/8ffd179bd4a445eaafb53d7accede863/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:41:27.527243 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44146f-fd19-4010-8abd-3b13a3884e4e None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:41:27.527532 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44146f-fd19-4010-8abd-3b13a3884e4e None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:41:27.527814 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44146f-fd19-4010-8abd-3b13a3884e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:41:27.547417 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-2a44146f-fd19-4010-8abd-3b13a3884e4e None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:41:27.548661 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-2a44146f-fd19-4010-8abd-3b13a3884e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:41:27.607140 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1340/2699] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:41:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:41:27.617159 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da39c38-0bfa-4e1a-bbcd-3be6fe980ec2 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:41:27.620285 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-9da39c38-0bfa-4e1a-bbcd-3be6fe980ec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:41:27.639238 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:41:27.675333 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da39c38-0bfa-4e1a-bbcd-3be6fe980ec2 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:41:27.679034 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4e3e2-7436-4c89-9660-ff7f41a11806 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:41:27.679647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4e3e2-7436-4c89-9660-ff7f41a11806 service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:41:27.679939 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4e3e2-7436-4c89-9660-ff7f41a11806 service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:41:27.680248 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4e3e2-7436-4c89-9660-ff7f41a11806 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:41:27.689663 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced4e3e2-7436-4c89-9660-ff7f41a11806 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:41:27.691109 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced4e3e2-7436-4c89-9660-ff7f41a11806 service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:41:27.701234 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1360/2700] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:41:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:41:33.348739 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf7c800-05d3-4f38-ba8b-44b649477e98 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:41:33.349312 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf7c800-05d3-4f38-ba8b-44b649477e98 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:41:33.349524 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf7c800-05d3-4f38-ba8b-44b649477e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:41:33.381461 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-abf7c800-05d3-4f38-ba8b-44b649477e98 None None] MFA Rules not processed for user `8ffd179bd4a445eaafb53d7accede863`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:41:33.382796 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-abf7c800-05d3-4f38-ba8b-44b649477e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:41:33.431884 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:41:33.455417 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1341/2701] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:41:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:41:33.460108 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4e3e2-7436-4c89-9660-ff7f41a11806 service neutron] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:41:33.481343 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4e3e2-7436-4c89-9660-ff7f41a11806 service neutron] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:41:33.481343 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066edb9b-8299-41b9-adda-e4acc93ff131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:41:33.481343 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066edb9b-8299-41b9-adda-e4acc93ff131 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:41:33.481343 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066edb9b-8299-41b9-adda-e4acc93ff131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:41:33.481753 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066edb9b-8299-41b9-adda-e4acc93ff131 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:41:33.481753 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066edb9b-8299-41b9-adda-e4acc93ff131 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:41:33.482507 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066edb9b-8299-41b9-adda-e4acc93ff131 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:41:33.596539 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1361/2702] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:41:33 2022] POST /identity/v3/projects => generated 385 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:41:33.600586 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf7c800-05d3-4f38-ba8b-44b649477e98 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:41:33.619151 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf7c800-05d3-4f38-ba8b-44b649477e98 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:41:33.687449 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ceb7d6-d461-4d4d-9d9d-6ada61dce64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:41:33.688213 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ceb7d6-d461-4d4d-9d9d-6ada61dce64e admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:41:33.688566 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ceb7d6-d461-4d4d-9d9d-6ada61dce64e admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:41:33.688911 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ceb7d6-d461-4d4d-9d9d-6ada61dce64e admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:41:33.689661 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ceb7d6-d461-4d4d-9d9d-6ada61dce64e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:41:33.691740 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ceb7d6-d461-4d4d-9d9d-6ada61dce64e admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:41:33.772377 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1342/2703] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:41:33 2022] POST /identity/v3/users => generated 299 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:41:33.776538 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066edb9b-8299-41b9-adda-e4acc93ff131 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:41:33.788004 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066edb9b-8299-41b9-adda-e4acc93ff131 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:41:33.793097 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e13df-d4b4-4722-916f-d0fb1e587907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:41:33.793722 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e13df-d4b4-4722-916f-d0fb1e587907 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:41:33.794008 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e13df-d4b4-4722-916f-d0fb1e587907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:41:33.794301 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e13df-d4b4-4722-916f-d0fb1e587907 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:41:33.795043 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595e13df-d4b4-4722-916f-d0fb1e587907 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:41:33.796692 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595e13df-d4b4-4722-916f-d0fb1e587907 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:41:33.808642 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1362/2704] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:41:33.812815 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ceb7d6-d461-4d4d-9d9d-6ada61dce64e admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:41:33.823917 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ceb7d6-d461-4d4d-9d9d-6ada61dce64e admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:41:33.829674 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de88f566-5202-42ea-b8ca-ac04e97e4858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:41:33.830380 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88f566-5202-42ea-b8ca-ac04e97e4858 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:41:33.830666 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88f566-5202-42ea-b8ca-ac04e97e4858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:41:33.830946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88f566-5202-42ea-b8ca-ac04e97e4858 admin admin] PATH_INFO: `/v3/projects/9cb1f083fca24a3a8bfaa6b085b02856/users/acf6022c12a14426a39fdd7b9e9cebb1/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:41:33.849757 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de88f566-5202-42ea-b8ca-ac04e97e4858 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cb1f083fca24a3a8bfaa6b085b02856, user_id=acf6022c12a14426a39fdd7b9e9cebb1, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:41:33.852666 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de88f566-5202-42ea-b8ca-ac04e97e4858 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:41:33.932430 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1343/2705] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:41:33 2022] PUT /identity/v3/projects/9cb1f083fca24a3a8bfaa6b085b02856/users/acf6022c12a14426a39fdd7b9e9cebb1/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:41:33.936541 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e13df-d4b4-4722-916f-d0fb1e587907 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:41:33.947324 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e13df-d4b4-4722-916f-d0fb1e587907 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:41:34.006786 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bbc7c0-8d55-423d-a6b8-85acdb2408a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:41:34.007394 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bbc7c0-8d55-423d-a6b8-85acdb2408a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:41:34.007678 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bbc7c0-8d55-423d-a6b8-85acdb2408a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:41:34.007957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bbc7c0-8d55-423d-a6b8-85acdb2408a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:41:34.008698 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bbc7c0-8d55-423d-a6b8-85acdb2408a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:41:34.010914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bbc7c0-8d55-423d-a6b8-85acdb2408a0 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:41:34.023904 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1363/2706] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:41:34.028363 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de88f566-5202-42ea-b8ca-ac04e97e4858 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:41:34.039296 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de88f566-5202-42ea-b8ca-ac04e97e4858 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:41:34.044037 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988bcdd9-e1c9-40b0-b135-9eb81b1b012c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:41:34.044763 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988bcdd9-e1c9-40b0-b135-9eb81b1b012c admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:41:34.045052 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988bcdd9-e1c9-40b0-b135-9eb81b1b012c admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:41:34.045336 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988bcdd9-e1c9-40b0-b135-9eb81b1b012c admin admin] PATH_INFO: `/v3/projects/9cb1f083fca24a3a8bfaa6b085b02856/users/acf6022c12a14426a39fdd7b9e9cebb1/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:41:34.047355 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988bcdd9-e1c9-40b0-b135-9eb81b1b012c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cb1f083fca24a3a8bfaa6b085b02856, user_id=acf6022c12a14426a39fdd7b9e9cebb1, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:41:34.050028 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988bcdd9-e1c9-40b0-b135-9eb81b1b012c admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:41:34.061481 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1344/2707] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:41:34 2022] PUT /identity/v3/projects/9cb1f083fca24a3a8bfaa6b085b02856/users/acf6022c12a14426a39fdd7b9e9cebb1/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:41:39.754739 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1f918-132c-4d34-b86f-6703f823dacc None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:41:39.755231 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1f918-132c-4d34-b86f-6703f823dacc None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:41:39.755344 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1f918-132c-4d34-b86f-6703f823dacc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:41:39.788699 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-66f1f918-132c-4d34-b86f-6703f823dacc None None] MFA Rules not processed for user `acf6022c12a14426a39fdd7b9e9cebb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:41:39.790773 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-66f1f918-132c-4d34-b86f-6703f823dacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:41:39.879222 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1364/2708] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:41:39 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:41:39.891534 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988bcdd9-e1c9-40b0-b135-9eb81b1b012c admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:41:39.897822 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-988bcdd9-e1c9-40b0-b135-9eb81b1b012c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:41:39.979926 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988bcdd9-e1c9-40b0-b135-9eb81b1b012c admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:41:39.986173 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6bbc13-6e3c-42fb-b1b7-809ddaf57312 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c386fa0483cf42efaddbc2395f6d1fce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:41:39.986173 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6bbc13-6e3c-42fb-b1b7-809ddaf57312 service glance] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:41:39.986173 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6bbc13-6e3c-42fb-b1b7-809ddaf57312 service glance] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:41:39.986173 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6bbc13-6e3c-42fb-b1b7-809ddaf57312 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:41:40.081442 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:41:40.091002 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6bbc13-6e3c-42fb-b1b7-809ddaf57312 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:41:40.092598 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6bbc13-6e3c-42fb-b1b7-809ddaf57312 service glance] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:41:40.105839 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1345/2709] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:41:39 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:41:40.365747 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5273a-d11a-4cab-af3a-de1bea806e72 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:41:40.367395 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5273a-d11a-4cab-af3a-de1bea806e72 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:41:40.367395 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5273a-d11a-4cab-af3a-de1bea806e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:41:40.380627 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:41:40.438664 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-54d5273a-d11a-4cab-af3a-de1bea806e72 None None] MFA Rules not processed for user `1654fb6a81e14d6b99ab495bb6d029d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:41:40.439923 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-54d5273a-d11a-4cab-af3a-de1bea806e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:41:40.521953 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1365/2710] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 5 15:41:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:41:40.688568 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6bbc13-6e3c-42fb-b1b7-809ddaf57312 service glance] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:41:40.690464 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-1f6bbc13-6e3c-42fb-b1b7-809ddaf57312 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:41:40.791085 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6bbc13-6e3c-42fb-b1b7-809ddaf57312 service glance] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:41:40.796333 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ddd67e-524d-4ebc-8dd4-34fa7e077023 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85154ce7446245738edcc8702dbf6861', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:41:40.797055 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ddd67e-524d-4ebc-8dd4-34fa7e077023 service swift] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:41:40.797548 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ddd67e-524d-4ebc-8dd4-34fa7e077023 service swift] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:41:40.798029 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ddd67e-524d-4ebc-8dd4-34fa7e077023 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:41:40.807173 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ddd67e-524d-4ebc-8dd4-34fa7e077023 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:41:40.808817 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ddd67e-524d-4ebc-8dd4-34fa7e077023 service swift] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:41:40.816924 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1346/2711] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 5 15:41:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 15:41:44.739035 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d5273a-d11a-4cab-af3a-de1bea806e72 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:41:44.740538 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-54d5273a-d11a-4cab-af3a-de1bea806e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:41:44.925792 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d5273a-d11a-4cab-af3a-de1bea806e72 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:41:44.935308 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec49597-a8ef-4af0-8faa-381fbfc7ecee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:41:44.936153 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec49597-a8ef-4af0-8faa-381fbfc7ecee service nova] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:41:44.936435 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec49597-a8ef-4af0-8faa-381fbfc7ecee service nova] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:41:44.936718 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec49597-a8ef-4af0-8faa-381fbfc7ecee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:41:44.939382 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-5ec49597-a8ef-4af0-8faa-381fbfc7ecee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:41:44.973310 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec49597-a8ef-4af0-8faa-381fbfc7ecee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:41:44.975960 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec49597-a8ef-4af0-8faa-381fbfc7ecee service nova] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:41:45.011187 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1366/2712] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:41:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:41:50.813670 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=45 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:41:50.880898 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5fc0fc-94fb-4487-b8cc-d1e4619fb960 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:41:50.924300 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5fc0fc-94fb-4487-b8cc-d1e4619fb960 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:41:50.924300 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5fc0fc-94fb-4487-b8cc-d1e4619fb960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:41:50.924300 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-fa5fc0fc-94fb-4487-b8cc-d1e4619fb960 None None] MFA Rules not processed for user `dd5c43d63a754eeaaba429f5c8153bba`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:41:50.924300 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-fa5fc0fc-94fb-4487-b8cc-d1e4619fb960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:41:50.957843 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1347/2713] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:41:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:41:51.265432 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec49597-a8ef-4af0-8faa-381fbfc7ecee service nova] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:41:51.266863 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-5ec49597-a8ef-4af0-8faa-381fbfc7ecee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:41:51.322066 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec49597-a8ef-4af0-8faa-381fbfc7ecee service nova] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:41:51.325578 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf78638-6b67-4c6e-8a87-2f5009925000 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:41:51.326174 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf78638-6b67-4c6e-8a87-2f5009925000 service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:41:51.326457 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf78638-6b67-4c6e-8a87-2f5009925000 service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:41:51.326728 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf78638-6b67-4c6e-8a87-2f5009925000 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:41:51.328473 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-ccf78638-6b67-4c6e-8a87-2f5009925000 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:41:51.338579 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf78638-6b67-4c6e-8a87-2f5009925000 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:41:51.340133 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf78638-6b67-4c6e-8a87-2f5009925000 service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:41:51.345557 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=100 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:41:51.347284 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb7f1b-e51c-4671-bc5a-aab1cc1051c3 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:41:51.347715 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb7f1b-e51c-4671-bc5a-aab1cc1051c3 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:41:51.348044 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb7f1b-e51c-4671-bc5a-aab1cc1051c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:41:51.349353 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1367/2714] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:41:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:41:51.371041 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-60fb7f1b-e51c-4671-bc5a-aab1cc1051c3 None None] MFA Rules not processed for user `8ffd179bd4a445eaafb53d7accede863`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:41:51.372318 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-60fb7f1b-e51c-4671-bc5a-aab1cc1051c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:41:51.379656 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1348/2715] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:41:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:41:51.708266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf78638-6b67-4c6e-8a87-2f5009925000 service neutron] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:41:51.717936 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf78638-6b67-4c6e-8a87-2f5009925000 service neutron] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:41:51.721868 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63960c31-58d7-431f-be58-0fcf6c55a09e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:41:51.722475 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63960c31-58d7-431f-be58-0fcf6c55a09e service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:41:51.722758 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63960c31-58d7-431f-be58-0fcf6c55a09e service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:41:51.723028 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63960c31-58d7-431f-be58-0fcf6c55a09e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:41:51.724770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-63960c31-58d7-431f-be58-0fcf6c55a09e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:41:51.735560 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63960c31-58d7-431f-be58-0fcf6c55a09e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:41:51.737028 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63960c31-58d7-431f-be58-0fcf6c55a09e service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:41:51.745151 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1368/2716] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:41:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:41:51.833353 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb7f1b-e51c-4671-bc5a-aab1cc1051c3 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:41:51.834789 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-60fb7f1b-e51c-4671-bc5a-aab1cc1051c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:41:51.907930 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb7f1b-e51c-4671-bc5a-aab1cc1051c3 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:41:51.912933 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db8dddb-6e6b-4f22-9f19-7ea95d4dbce2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '107a7c18a2c6440b86e1b7c3e39cc724', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:41:51.913551 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db8dddb-6e6b-4f22-9f19-7ea95d4dbce2 service placement] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:41:51.913831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db8dddb-6e6b-4f22-9f19-7ea95d4dbce2 service placement] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:41:51.914105 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db8dddb-6e6b-4f22-9f19-7ea95d4dbce2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:41:51.916495 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-2db8dddb-6e6b-4f22-9f19-7ea95d4dbce2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:41:51.928291 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db8dddb-6e6b-4f22-9f19-7ea95d4dbce2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:41:51.929869 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db8dddb-6e6b-4f22-9f19-7ea95d4dbce2 service placement] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:41:51.945114 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1349/2717] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:41:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:41:54.792696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63960c31-58d7-431f-be58-0fcf6c55a09e service neutron] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:41:54.814336 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63960c31-58d7-431f-be58-0fcf6c55a09e service neutron] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:41:54.823760 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0806e7aa-2d86-485c-afa5-41dfdf118e26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:41:54.824527 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806e7aa-2d86-485c-afa5-41dfdf118e26 service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:41:54.824814 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806e7aa-2d86-485c-afa5-41dfdf118e26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:41:54.825195 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806e7aa-2d86-485c-afa5-41dfdf118e26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:41:54.827237 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-0806e7aa-2d86-485c-afa5-41dfdf118e26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:41:54.837843 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0806e7aa-2d86-485c-afa5-41dfdf118e26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:41:54.839389 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0806e7aa-2d86-485c-afa5-41dfdf118e26 service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:41:54.849428 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1369/2718] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:41:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:41:55.188484 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad879b-d17a-437f-9bdc-5300046d3a55 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:41:55.189421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad879b-d17a-437f-9bdc-5300046d3a55 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:41:55.189904 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad879b-d17a-437f-9bdc-5300046d3a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:41:55.211257 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-39ad879b-d17a-437f-9bdc-5300046d3a55 None None] MFA Rules not processed for user `1654fb6a81e14d6b99ab495bb6d029d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:41:55.217134 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-39ad879b-d17a-437f-9bdc-5300046d3a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:41:55.221182 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1350/2719] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 5 15:41:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:41:55.235478 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0806e7aa-2d86-485c-afa5-41dfdf118e26 service neutron] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:41:55.248822 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0806e7aa-2d86-485c-afa5-41dfdf118e26 service neutron] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:41:55.253945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a57b71c-7331-4579-aa8c-c5e5bd2a0440 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85154ce7446245738edcc8702dbf6861', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:41:55.254562 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57b71c-7331-4579-aa8c-c5e5bd2a0440 service swift] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:41:55.254854 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57b71c-7331-4579-aa8c-c5e5bd2a0440 service swift] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:41:55.255132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57b71c-7331-4579-aa8c-c5e5bd2a0440 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:41:55.263557 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a57b71c-7331-4579-aa8c-c5e5bd2a0440 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:41:55.265201 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a57b71c-7331-4579-aa8c-c5e5bd2a0440 service swift] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:41:55.273696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1370/2720] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 5 15:41:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 15:41:56.771334 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad879b-d17a-437f-9bdc-5300046d3a55 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:41:56.788142 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad879b-d17a-437f-9bdc-5300046d3a55 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:41:56.795293 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ed152-e0d0-457b-879a-3f46c1d94565 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:41:56.796385 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ed152-e0d0-457b-879a-3f46c1d94565 service nova] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:41:56.797101 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ed152-e0d0-457b-879a-3f46c1d94565 service nova] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:41:56.797661 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ed152-e0d0-457b-879a-3f46c1d94565 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:41:56.801241 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-718ed152-e0d0-457b-879a-3f46c1d94565 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:41:56.815730 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ed152-e0d0-457b-879a-3f46c1d94565 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:41:56.817519 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ed152-e0d0-457b-879a-3f46c1d94565 service nova] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:41:56.832223 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1351/2721] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:41:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:42:05.273952 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=18 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:42:06.826689 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=69 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:42:10.080787 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a57b71c-7331-4579-aa8c-c5e5bd2a0440 service swift] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:42:10.092917 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a57b71c-7331-4579-aa8c-c5e5bd2a0440 service swift] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:42:10.100586 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9354a52-d2a3-4fe9-a072-15889d9c6c4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0497dab1f44bb7b8196d39bf01d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:42:10.101260 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9354a52-d2a3-4fe9-a072-15889d9c6c4a service nova] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:42:10.101548 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9354a52-d2a3-4fe9-a072-15889d9c6c4a service nova] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:42:10.101828 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9354a52-d2a3-4fe9-a072-15889d9c6c4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:42:10.104711 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-d9354a52-d2a3-4fe9-a072-15889d9c6c4a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:42:10.118128 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9354a52-d2a3-4fe9-a072-15889d9c6c4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:42:10.121415 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9354a52-d2a3-4fe9-a072-15889d9c6c4a service nova] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:42:10.131346 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1371/2722] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:42:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:42:18.738420 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df36d76-5972-451c-ad42-47b4ddf92564 None None] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:42:18.738999 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df36d76-5972-451c-ad42-47b4ddf92564 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:42:18.738999 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df36d76-5972-451c-ad42-47b4ddf92564 None None] PATH_INFO: `/` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:42:18.740508 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1352/2723] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 5 15:42:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 15:42:18.747312 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82752e-9fbc-4772-84ad-77c283765565 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:42:18.747802 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82752e-9fbc-4772-84ad-77c283765565 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:42:18.748339 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82752e-9fbc-4772-84ad-77c283765565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:42:18.792204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-ee82752e-9fbc-4772-84ad-77c283765565 None None] MFA Rules not processed for user `d13a1638b2ca497fb02c71615559d1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:42:18.793686 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-ee82752e-9fbc-4772-84ad-77c283765565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:42:18.800851 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1372/2724] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 5 15:42:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:42:18.814075 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df36d76-5972-451c-ad42-47b4ddf92564 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:42:18.826825 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df36d76-5972-451c-ad42-47b4ddf92564 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:42:18.830559 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9407466-6652-4657-af72-55951ca1a444 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:42:18.831165 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9407466-6652-4657-af72-55951ca1a444 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:42:18.831495 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9407466-6652-4657-af72-55951ca1a444 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:42:18.831774 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9407466-6652-4657-af72-55951ca1a444 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:42:18.839003 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9407466-6652-4657-af72-55951ca1a444 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:42:18.840482 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9407466-6652-4657-af72-55951ca1a444 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:42:18.848852 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1353/2725] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:42:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:42:18.931478 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676067f2-6098-4969-8371-f7490b447876 None None] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:42:18.934847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676067f2-6098-4969-8371-f7490b447876 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:42:18.934847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676067f2-6098-4969-8371-f7490b447876 None None] PATH_INFO: `/` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:42:18.934847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1373/2726] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 5 15:42:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 15:42:18.938555 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71791cd4-7c35-4465-bb3a-016b435ebdf4 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:42:18.938856 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71791cd4-7c35-4465-bb3a-016b435ebdf4 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:42:18.939162 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71791cd4-7c35-4465-bb3a-016b435ebdf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:42:18.982051 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-71791cd4-7c35-4465-bb3a-016b435ebdf4 None None] MFA Rules not processed for user `bc0497dab1f44bb7b8196d39bf01d770`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:42:18.983316 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-71791cd4-7c35-4465-bb3a-016b435ebdf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:42:18.992688 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1354/2727] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 5 15:42:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:42:19.003803 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676067f2-6098-4969-8371-f7490b447876 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:42:19.015579 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676067f2-6098-4969-8371-f7490b447876 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:42:19.019088 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7d5e64-e45d-49ab-a984-3d2690487df9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:42:19.019689 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d5e64-e45d-49ab-a984-3d2690487df9 service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:42:19.019970 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d5e64-e45d-49ab-a984-3d2690487df9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:42:19.020278 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d5e64-e45d-49ab-a984-3d2690487df9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:42:19.030189 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7d5e64-e45d-49ab-a984-3d2690487df9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:42:19.031877 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7d5e64-e45d-49ab-a984-3d2690487df9 service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:42:19.042907 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1374/2728] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:42:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:42:25.891198 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71791cd4-7c35-4465-bb3a-016b435ebdf4 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:42:25.901007 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71791cd4-7c35-4465-bb3a-016b435ebdf4 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:42:25.904799 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d461e-6c4a-498e-9c61-6da8684718a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:42:25.905465 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d461e-6c4a-498e-9c61-6da8684718a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:42:25.905749 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d461e-6c4a-498e-9c61-6da8684718a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:42:25.906031 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d461e-6c4a-498e-9c61-6da8684718a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:42:25.916226 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38d461e-6c4a-498e-9c61-6da8684718a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:42:25.917683 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38d461e-6c4a-498e-9c61-6da8684718a4 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:42:25.927089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1355/2729] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:42:25 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:42:29.037988 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=21 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:42:35.922905 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=21 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:43:06.701985 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7d5e64-e45d-49ab-a984-3d2690487df9 service neutron] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:06.703389 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-be7d5e64-e45d-49ab-a984-3d2690487df9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:06.794532 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7d5e64-e45d-49ab-a984-3d2690487df9 service neutron] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:06.798190 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d2469-2d6a-4e24-b271-dc572cc0c09b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:06.798807 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d2469-2d6a-4e24-b271-dc572cc0c09b service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:06.799097 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d2469-2d6a-4e24-b271-dc572cc0c09b service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:06.799383 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d2469-2d6a-4e24-b271-dc572cc0c09b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:06.801164 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-778d2469-2d6a-4e24-b271-dc572cc0c09b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:06.812097 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d2469-2d6a-4e24-b271-dc572cc0c09b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:06.813531 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d2469-2d6a-4e24-b271-dc572cc0c09b service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:06.822347 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1375/2730] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:43:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:43:10.814519 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d461e-6c4a-498e-9c61-6da8684718a4 service neutron] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:10.823089 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d461e-6c4a-498e-9c61-6da8684718a4 service neutron] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:10.827244 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a95ac-5abd-46af-9e9f-d33cbab0a128 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:10.827839 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a95ac-5abd-46af-9e9f-d33cbab0a128 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:10.828148 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a95ac-5abd-46af-9e9f-d33cbab0a128 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:10.828429 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a95ac-5abd-46af-9e9f-d33cbab0a128 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:10.830143 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-bd6a95ac-5abd-46af-9e9f-d33cbab0a128 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:10.891272 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6a95ac-5abd-46af-9e9f-d33cbab0a128 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:10.892959 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6a95ac-5abd-46af-9e9f-d33cbab0a128 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:10.908526 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1356/2731] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:43:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:43:14.101055 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d2469-2d6a-4e24-b271-dc572cc0c09b service cinder] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:14.103117 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-778d2469-2d6a-4e24-b271-dc572cc0c09b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:14.117631 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d2469-2d6a-4e24-b271-dc572cc0c09b service cinder] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:14.123208 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a02b7-ad21-4e66-919c-43f4a5ee1769 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '107a7c18a2c6440b86e1b7c3e39cc724', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:14.125659 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a02b7-ad21-4e66-919c-43f4a5ee1769 service placement] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:14.125659 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a02b7-ad21-4e66-919c-43f4a5ee1769 service placement] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:14.125659 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a02b7-ad21-4e66-919c-43f4a5ee1769 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:14.126302 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-0b9a02b7-ad21-4e66-919c-43f4a5ee1769 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:14.145323 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9a02b7-ad21-4e66-919c-43f4a5ee1769 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:14.146947 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9a02b7-ad21-4e66-919c-43f4a5ee1769 service placement] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:14.157202 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1376/2732] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:43:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:43:14.228088 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a95ac-5abd-46af-9e9f-d33cbab0a128 service neutron] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:14.229645 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-bd6a95ac-5abd-46af-9e9f-d33cbab0a128 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:14.242270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a95ac-5abd-46af-9e9f-d33cbab0a128 service neutron] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:14.247574 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e52a2-5b5a-4400-9e42-9d9c3a5433a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:14.248201 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e52a2-5b5a-4400-9e42-9d9c3a5433a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:14.248485 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e52a2-5b5a-4400-9e42-9d9c3a5433a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:14.248765 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e52a2-5b5a-4400-9e42-9d9c3a5433a6 admin admin] PATH_INFO: `/v3/users/dd5c43d63a754eeaaba429f5c8153bba` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:14.249982 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271e52a2-5b5a-4400-9e42-9d9c3a5433a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd5c43d63a754eeaaba429f5c8153bba)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:14.251874 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271e52a2-5b5a-4400-9e42-9d9c3a5433a6 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:14.403871 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-271e52a2-5b5a-4400-9e42-9d9c3a5433a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd5c43d63a754eeaaba429f5c8153bba'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:14.411173 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-271e52a2-5b5a-4400-9e42-9d9c3a5433a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd5c43d63a754eeaaba429f5c8153bba'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:14.411844 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-271e52a2-5b5a-4400-9e42-9d9c3a5433a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd5c43d63a754eeaaba429f5c8153bba'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:14.424251 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-271e52a2-5b5a-4400-9e42-9d9c3a5433a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd5c43d63a754eeaaba429f5c8153bba'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:14.424823 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-271e52a2-5b5a-4400-9e42-9d9c3a5433a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd5c43d63a754eeaaba429f5c8153bba'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:14.490155 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1357/2733] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:43:14 2022] DELETE /identity/v3/users/dd5c43d63a754eeaaba429f5c8153bba => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:43:14.747358 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a02b7-ad21-4e66-919c-43f4a5ee1769 service placement] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:14.758302 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a02b7-ad21-4e66-919c-43f4a5ee1769 service placement] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:14.817470 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb63543f-9b72-45e3-b32a-40feddf0eea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:14.818103 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63543f-9b72-45e3-b32a-40feddf0eea8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:14.818387 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63543f-9b72-45e3-b32a-40feddf0eea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:14.818660 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63543f-9b72-45e3-b32a-40feddf0eea8 admin admin] PATH_INFO: `/v3/projects/2c77da48edba4903a1c51fee2a856167` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:14.819819 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb63543f-9b72-45e3-b32a-40feddf0eea8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c77da48edba4903a1c51fee2a856167)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:14.821775 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb63543f-9b72-45e3-b32a-40feddf0eea8 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:14.921828 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-bb63543f-9b72-45e3-b32a-40feddf0eea8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c77da48edba4903a1c51fee2a856167', 'request_id': 'req-bb63543f-9b72-45e3-b32a-40feddf0eea8'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:14.927793 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-bb63543f-9b72-45e3-b32a-40feddf0eea8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c77da48edba4903a1c51fee2a856167', 'request_id': 'req-bb63543f-9b72-45e3-b32a-40feddf0eea8'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:14.933345 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-bb63543f-9b72-45e3-b32a-40feddf0eea8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c77da48edba4903a1c51fee2a856167', 'request_id': 'req-bb63543f-9b72-45e3-b32a-40feddf0eea8'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:14.986244 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:43:14.989992 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-bb63543f-9b72-45e3-b32a-40feddf0eea8 admin admin] The token cache is being invalidate because project 2c77da48edba4903a1c51fee2a856167 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:43:14.990227 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-bb63543f-9b72-45e3-b32a-40feddf0eea8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:14.990840 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-bb63543f-9b72-45e3-b32a-40feddf0eea8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:14.992421 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1377/2734] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:43:14 2022] DELETE /identity/v3/projects/2c77da48edba4903a1c51fee2a856167 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:43:14.997879 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba8d31d-d2f2-4e22-9092-77d2b5561af2 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:14.998167 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba8d31d-d2f2-4e22-9092-77d2b5561af2 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:14.998448 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba8d31d-d2f2-4e22-9092-77d2b5561af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:15.018005 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-3ba8d31d-d2f2-4e22-9092-77d2b5561af2 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:43:15.019202 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-3ba8d31d-d2f2-4e22-9092-77d2b5561af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:15.077018 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1358/2735] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:43:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:43:15.084845 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85575480-7497-4b92-9ddf-92f397fde4b9 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:15.085136 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85575480-7497-4b92-9ddf-92f397fde4b9 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:15.085414 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85575480-7497-4b92-9ddf-92f397fde4b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:15.105566 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-85575480-7497-4b92-9ddf-92f397fde4b9 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:43:15.106762 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-85575480-7497-4b92-9ddf-92f397fde4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:15.115020 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1378/2736] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:43:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:43:15.119216 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba8d31d-d2f2-4e22-9092-77d2b5561af2 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:15.129489 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba8d31d-d2f2-4e22-9092-77d2b5561af2 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:15.134017 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e16bc-fc16-4cca-8d33-23721f6eecc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:15.134616 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e16bc-fc16-4cca-8d33-23721f6eecc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:15.134897 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e16bc-fc16-4cca-8d33-23721f6eecc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:15.135172 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e16bc-fc16-4cca-8d33-23721f6eecc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:15.135862 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921e16bc-fc16-4cca-8d33-23721f6eecc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:43:15.136209 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921e16bc-fc16-4cca-8d33-23721f6eecc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:15.138402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921e16bc-fc16-4cca-8d33-23721f6eecc9 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:15.139829 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:43:15.150246 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1359/2737] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:43:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:43:15.154451 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85575480-7497-4b92-9ddf-92f397fde4b9 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:15.164091 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85575480-7497-4b92-9ddf-92f397fde4b9 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:15.169656 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db33977d-dc9d-4870-aa60-ead95faa3fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:15.170876 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33977d-dc9d-4870-aa60-ead95faa3fb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:15.170876 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33977d-dc9d-4870-aa60-ead95faa3fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:15.170876 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33977d-dc9d-4870-aa60-ead95faa3fb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:15.171533 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db33977d-dc9d-4870-aa60-ead95faa3fb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:15.173482 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db33977d-dc9d-4870-aa60-ead95faa3fb3 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:15.239559 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1379/2738] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:43:15 2022] POST /identity/v3/projects => generated 367 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:43:15.243717 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e16bc-fc16-4cca-8d33-23721f6eecc9 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:15.254859 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e16bc-fc16-4cca-8d33-23721f6eecc9 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:15.311326 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef9c533-cf92-4d80-b21b-8cd4da641c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:15.311946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9c533-cf92-4d80-b21b-8cd4da641c9c admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:15.312263 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9c533-cf92-4d80-b21b-8cd4da641c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:15.312542 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9c533-cf92-4d80-b21b-8cd4da641c9c admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:15.313207 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef9c533-cf92-4d80-b21b-8cd4da641c9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:15.315115 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef9c533-cf92-4d80-b21b-8cd4da641c9c admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:15.425725 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1360/2739] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:43:15 2022] POST /identity/v3/users => generated 284 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:43:15.429746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db33977d-dc9d-4870-aa60-ead95faa3fb3 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:15.440866 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db33977d-dc9d-4870-aa60-ead95faa3fb3 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:15.445508 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2b3d24-bc73-45d8-b841-3b25091e982e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:15.446112 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b3d24-bc73-45d8-b841-3b25091e982e admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:15.446393 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b3d24-bc73-45d8-b841-3b25091e982e admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:15.446666 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b3d24-bc73-45d8-b841-3b25091e982e admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:15.447390 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2b3d24-bc73-45d8-b841-3b25091e982e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:15.449005 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2b3d24-bc73-45d8-b841-3b25091e982e admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:15.459309 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1380/2740] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:43:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:43:15.463058 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef9c533-cf92-4d80-b21b-8cd4da641c9c admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:15.473468 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef9c533-cf92-4d80-b21b-8cd4da641c9c admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:15.478132 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8302571-fbc5-406a-8eeb-8434937ad74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:15.478723 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8302571-fbc5-406a-8eeb-8434937ad74e admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:15.479005 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8302571-fbc5-406a-8eeb-8434937ad74e admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:15.479278 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8302571-fbc5-406a-8eeb-8434937ad74e admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:15.479957 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8302571-fbc5-406a-8eeb-8434937ad74e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:15.481630 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8302571-fbc5-406a-8eeb-8434937ad74e admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:15.493002 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1361/2741] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:43:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 15:43:15.497086 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2b3d24-bc73-45d8-b841-3b25091e982e admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:15.506699 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2b3d24-bc73-45d8-b841-3b25091e982e admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:15.511234 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf239f-d06a-4cc7-8e11-e06f2dca4d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:15.511879 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf239f-d06a-4cc7-8e11-e06f2dca4d15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:15.512708 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf239f-d06a-4cc7-8e11-e06f2dca4d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:15.512992 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf239f-d06a-4cc7-8e11-e06f2dca4d15 admin admin] PATH_INFO: `/v3/projects/d1f886553c164fec995e15a34e5676c7/users/b1d5bd74bbcf415e9241e5e05431a074/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:15.526973 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbf239f-d06a-4cc7-8e11-e06f2dca4d15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1f886553c164fec995e15a34e5676c7, user_id=b1d5bd74bbcf415e9241e5e05431a074, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:15.530204 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbf239f-d06a-4cc7-8e11-e06f2dca4d15 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:15.619356 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1381/2742] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:43:15 2022] PUT /identity/v3/projects/d1f886553c164fec995e15a34e5676c7/users/b1d5bd74bbcf415e9241e5e05431a074/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:43:15.636227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f0e8f-5df8-4026-ba77-17caeae52529 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:15.636521 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f0e8f-5df8-4026-ba77-17caeae52529 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:15.636826 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f0e8f-5df8-4026-ba77-17caeae52529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:15.667962 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-a38f0e8f-5df8-4026-ba77-17caeae52529 None None] MFA Rules not processed for user `b1d5bd74bbcf415e9241e5e05431a074`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:43:15.669168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-a38f0e8f-5df8-4026-ba77-17caeae52529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:15.733938 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1362/2743] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:43:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:43:15.738344 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf239f-d06a-4cc7-8e11-e06f2dca4d15 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:15.799869 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf239f-d06a-4cc7-8e11-e06f2dca4d15 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:15.804576 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef8fa1-da11-42ae-879a-5234da6d6e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:15.805184 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef8fa1-da11-42ae-879a-5234da6d6e51 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:15.805472 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef8fa1-da11-42ae-879a-5234da6d6e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:15.805750 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef8fa1-da11-42ae-879a-5234da6d6e51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:15.806413 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ef8fa1-da11-42ae-879a-5234da6d6e51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:15.808329 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ef8fa1-da11-42ae-879a-5234da6d6e51 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:15.917357 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1382/2744] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:43:15 2022] POST /identity/v3/projects => generated 369 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:43:15.922139 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f0e8f-5df8-4026-ba77-17caeae52529 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:15.932897 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f0e8f-5df8-4026-ba77-17caeae52529 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:15.987915 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b4e155-a8e4-4282-83c9-85648c24ebad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:15.988577 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4e155-a8e4-4282-83c9-85648c24ebad admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:15.988862 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4e155-a8e4-4282-83c9-85648c24ebad admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:15.989139 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4e155-a8e4-4282-83c9-85648c24ebad admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:15.989842 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b4e155-a8e4-4282-83c9-85648c24ebad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:15.991738 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b4e155-a8e4-4282-83c9-85648c24ebad admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:16.001083 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:43:16.090581 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1363/2745] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:43:15 2022] POST /identity/v3/users => generated 291 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:43:16.094511 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef8fa1-da11-42ae-879a-5234da6d6e51 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:16.104835 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef8fa1-da11-42ae-879a-5234da6d6e51 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:16.110045 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3594a76-8a09-45b0-a372-09e8266af624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:16.110644 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3594a76-8a09-45b0-a372-09e8266af624 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:16.110922 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3594a76-8a09-45b0-a372-09e8266af624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:16.111201 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3594a76-8a09-45b0-a372-09e8266af624 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:16.111935 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3594a76-8a09-45b0-a372-09e8266af624 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:16.113575 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3594a76-8a09-45b0-a372-09e8266af624 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:16.125738 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1383/2746] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:43:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:43:16.129788 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b4e155-a8e4-4282-83c9-85648c24ebad admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:16.140308 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b4e155-a8e4-4282-83c9-85648c24ebad admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:16.144894 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c40a6-b30c-43ee-b429-73c430b584f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:16.145544 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c40a6-b30c-43ee-b429-73c430b584f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:16.145827 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c40a6-b30c-43ee-b429-73c430b584f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:16.146106 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c40a6-b30c-43ee-b429-73c430b584f0 admin admin] PATH_INFO: `/v3/projects/8dec0d78dbb3494983bfead5385ab10f/users/fda82a0ff36945a5b4f813a1bf56f09e/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:16.160814 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114c40a6-b30c-43ee-b429-73c430b584f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dec0d78dbb3494983bfead5385ab10f, user_id=fda82a0ff36945a5b4f813a1bf56f09e, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:16.163427 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114c40a6-b30c-43ee-b429-73c430b584f0 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:16.222045 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1364/2747] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:43:16 2022] PUT /identity/v3/projects/8dec0d78dbb3494983bfead5385ab10f/users/fda82a0ff36945a5b4f813a1bf56f09e/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:43:16.226006 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3594a76-8a09-45b0-a372-09e8266af624 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:16.236588 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3594a76-8a09-45b0-a372-09e8266af624 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:16.292647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c005efd7-e795-427d-b33a-ad33233e1da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:16.293262 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005efd7-e795-427d-b33a-ad33233e1da2 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:16.294170 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005efd7-e795-427d-b33a-ad33233e1da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:16.294450 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005efd7-e795-427d-b33a-ad33233e1da2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:16.295142 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c005efd7-e795-427d-b33a-ad33233e1da2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:16.296778 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c005efd7-e795-427d-b33a-ad33233e1da2 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:16.306321 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1384/2748] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:43:16 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:43:16.310792 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c40a6-b30c-43ee-b429-73c430b584f0 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:16.324704 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c40a6-b30c-43ee-b429-73c430b584f0 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:16.329370 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f49b5-0f8e-4d0d-96c0-d96be0a0d430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:16.330038 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f49b5-0f8e-4d0d-96c0-d96be0a0d430 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:16.330328 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f49b5-0f8e-4d0d-96c0-d96be0a0d430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:16.330608 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f49b5-0f8e-4d0d-96c0-d96be0a0d430 admin admin] PATH_INFO: `/v3/projects/8dec0d78dbb3494983bfead5385ab10f/users/fda82a0ff36945a5b4f813a1bf56f09e/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:16.332591 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f49b5-0f8e-4d0d-96c0-d96be0a0d430 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dec0d78dbb3494983bfead5385ab10f, user_id=fda82a0ff36945a5b4f813a1bf56f09e, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:16.335190 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f49b5-0f8e-4d0d-96c0-d96be0a0d430 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:16.345934 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1365/2749] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:43:16 2022] PUT /identity/v3/projects/8dec0d78dbb3494983bfead5385ab10f/users/fda82a0ff36945a5b4f813a1bf56f09e/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:43:16.357964 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b61ee9b-a25e-4a61-b7a5-85e4bb55f4b8 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:16.358255 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b61ee9b-a25e-4a61-b7a5-85e4bb55f4b8 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:16.358530 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b61ee9b-a25e-4a61-b7a5-85e4bb55f4b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:16.372290 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:43:16.385696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-0b61ee9b-a25e-4a61-b7a5-85e4bb55f4b8 None None] MFA Rules not processed for user `fda82a0ff36945a5b4f813a1bf56f09e`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:43:16.386910 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-0b61ee9b-a25e-4a61-b7a5-85e4bb55f4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:16.452914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1385/2750] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:43:16 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:43:16.463591 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f49b5-0f8e-4d0d-96c0-d96be0a0d430 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:16.465799 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-b89f49b5-0f8e-4d0d-96c0-d96be0a0d430 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:16.523289 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f49b5-0f8e-4d0d-96c0-d96be0a0d430 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:16.526910 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96acd9fd-8486-49b2-b858-9247523dd2f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:16.527510 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96acd9fd-8486-49b2-b858-9247523dd2f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:16.528335 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96acd9fd-8486-49b2-b858-9247523dd2f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:16.528651 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96acd9fd-8486-49b2-b858-9247523dd2f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:16.537965 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96acd9fd-8486-49b2-b858-9247523dd2f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:16.541901 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96acd9fd-8486-49b2-b858-9247523dd2f9 service cinder] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:16.549648 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1366/2751] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:43:16 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:43:19.179377 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b61ee9b-a25e-4a61-b7a5-85e4bb55f4b8 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:19.240498 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b61ee9b-a25e-4a61-b7a5-85e4bb55f4b8 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:19.245087 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86664d75-f3a0-4eae-aaa5-cad8e3ceb6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:19.245734 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86664d75-f3a0-4eae-aaa5-cad8e3ceb6a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:19.246024 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86664d75-f3a0-4eae-aaa5-cad8e3ceb6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:19.246304 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86664d75-f3a0-4eae-aaa5-cad8e3ceb6a4 admin admin] PATH_INFO: `/v3/users/b1d5bd74bbcf415e9241e5e05431a074` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:19.248156 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86664d75-f3a0-4eae-aaa5-cad8e3ceb6a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1d5bd74bbcf415e9241e5e05431a074)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:19.250088 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86664d75-f3a0-4eae-aaa5-cad8e3ceb6a4 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:19.400246 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-86664d75-f3a0-4eae-aaa5-cad8e3ceb6a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1d5bd74bbcf415e9241e5e05431a074'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:19.400899 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-86664d75-f3a0-4eae-aaa5-cad8e3ceb6a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1d5bd74bbcf415e9241e5e05431a074'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:19.413535 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-86664d75-f3a0-4eae-aaa5-cad8e3ceb6a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1d5bd74bbcf415e9241e5e05431a074'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:19.476450 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-86664d75-f3a0-4eae-aaa5-cad8e3ceb6a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1d5bd74bbcf415e9241e5e05431a074'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:19.484307 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-86664d75-f3a0-4eae-aaa5-cad8e3ceb6a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1d5bd74bbcf415e9241e5e05431a074'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:19.490336 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1386/2752] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:43:19 2022] DELETE /identity/v3/users/b1d5bd74bbcf415e9241e5e05431a074 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:43:19.494863 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45bd751-cdc9-4b20-85f1-84164c845829 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:19.495356 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45bd751-cdc9-4b20-85f1-84164c845829 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:19.495457 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45bd751-cdc9-4b20-85f1-84164c845829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:19.513795 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-f45bd751-cdc9-4b20-85f1-84164c845829 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:43:19.515003 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-f45bd751-cdc9-4b20-85f1-84164c845829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:19.572323 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1367/2753] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:43:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:43:19.581985 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86664d75-f3a0-4eae-aaa5-cad8e3ceb6a4 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:19.583393 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-86664d75-f3a0-4eae-aaa5-cad8e3ceb6a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:19.597161 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:43:19.641272 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86664d75-f3a0-4eae-aaa5-cad8e3ceb6a4 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:19.644959 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb0194-e8c7-40ee-9f6f-7d422d7a8c6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:19.645574 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb0194-e8c7-40ee-9f6f-7d422d7a8c6f service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:19.645857 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb0194-e8c7-40ee-9f6f-7d422d7a8c6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:19.646133 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb0194-e8c7-40ee-9f6f-7d422d7a8c6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:19.655620 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdb0194-e8c7-40ee-9f6f-7d422d7a8c6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:19.657148 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdb0194-e8c7-40ee-9f6f-7d422d7a8c6f service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:19.667185 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1387/2754] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:43:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:43:20.045999 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45bd751-cdc9-4b20-85f1-84164c845829 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:20.049256 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-f45bd751-cdc9-4b20-85f1-84164c845829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:20.068793 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45bd751-cdc9-4b20-85f1-84164c845829 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:20.074308 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc669f6-9797-48cb-9760-6ba93cba539b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:20.075022 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc669f6-9797-48cb-9760-6ba93cba539b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:20.076132 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc669f6-9797-48cb-9760-6ba93cba539b admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:20.076500 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc669f6-9797-48cb-9760-6ba93cba539b admin admin] PATH_INFO: `/v3/projects/d1f886553c164fec995e15a34e5676c7` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:20.077809 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc669f6-9797-48cb-9760-6ba93cba539b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1f886553c164fec995e15a34e5676c7)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:20.080710 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc669f6-9797-48cb-9760-6ba93cba539b admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:20.188299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-6fc669f6-9797-48cb-9760-6ba93cba539b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1f886553c164fec995e15a34e5676c7', 'request_id': 'req-6fc669f6-9797-48cb-9760-6ba93cba539b'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:20.190247 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:INSERT=4,keystone:SELECT=92,keystone:DELETE=4 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:43:20.194817 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-6fc669f6-9797-48cb-9760-6ba93cba539b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1f886553c164fec995e15a34e5676c7', 'request_id': 'req-6fc669f6-9797-48cb-9760-6ba93cba539b'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:20.273018 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-6fc669f6-9797-48cb-9760-6ba93cba539b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1f886553c164fec995e15a34e5676c7', 'request_id': 'req-6fc669f6-9797-48cb-9760-6ba93cba539b'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:20.283638 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-6fc669f6-9797-48cb-9760-6ba93cba539b admin admin] The token cache is being invalidate because project d1f886553c164fec995e15a34e5676c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:43:20.283921 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-6fc669f6-9797-48cb-9760-6ba93cba539b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:20.284567 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-6fc669f6-9797-48cb-9760-6ba93cba539b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:20.286066 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1368/2755] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:43:20 2022] DELETE /identity/v3/projects/d1f886553c164fec995e15a34e5676c7 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:43:20.290251 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb0194-e8c7-40ee-9f6f-7d422d7a8c6f service neutron] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:20.299993 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb0194-e8c7-40ee-9f6f-7d422d7a8c6f service neutron] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:20.361373 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da19092-4c1f-431c-a052-9f033fa70d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:20.362009 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da19092-4c1f-431c-a052-9f033fa70d04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:20.362293 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da19092-4c1f-431c-a052-9f033fa70d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:20.362570 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da19092-4c1f-431c-a052-9f033fa70d04 admin admin] PATH_INFO: `/v3/users/fda82a0ff36945a5b4f813a1bf56f09e` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:20.363750 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da19092-4c1f-431c-a052-9f033fa70d04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fda82a0ff36945a5b4f813a1bf56f09e)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:20.365687 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da19092-4c1f-431c-a052-9f033fa70d04 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:20.531531 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-9da19092-4c1f-431c-a052-9f033fa70d04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fda82a0ff36945a5b4f813a1bf56f09e'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:20.532235 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-9da19092-4c1f-431c-a052-9f033fa70d04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fda82a0ff36945a5b4f813a1bf56f09e'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:20.545077 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-9da19092-4c1f-431c-a052-9f033fa70d04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fda82a0ff36945a5b4f813a1bf56f09e'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:20.603720 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-9da19092-4c1f-431c-a052-9f033fa70d04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fda82a0ff36945a5b4f813a1bf56f09e'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:20.611499 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-9da19092-4c1f-431c-a052-9f033fa70d04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fda82a0ff36945a5b4f813a1bf56f09e'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:20.617555 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1388/2756] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:43:20 2022] DELETE /identity/v3/users/fda82a0ff36945a5b4f813a1bf56f09e => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:43:21.005392 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc669f6-9797-48cb-9760-6ba93cba539b admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:21.007008 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-6fc669f6-9797-48cb-9760-6ba93cba539b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:21.022153 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc669f6-9797-48cb-9760-6ba93cba539b admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:21.079282 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea582629-19f6-4cfb-8f91-d501f5f8f0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:21.080004 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea582629-19f6-4cfb-8f91-d501f5f8f0dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:21.080325 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea582629-19f6-4cfb-8f91-d501f5f8f0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:21.080607 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea582629-19f6-4cfb-8f91-d501f5f8f0dc admin admin] PATH_INFO: `/v3/projects/8dec0d78dbb3494983bfead5385ab10f` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:21.081791 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea582629-19f6-4cfb-8f91-d501f5f8f0dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dec0d78dbb3494983bfead5385ab10f)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:21.083772 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea582629-19f6-4cfb-8f91-d501f5f8f0dc admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:21.183993 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-ea582629-19f6-4cfb-8f91-d501f5f8f0dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dec0d78dbb3494983bfead5385ab10f', 'request_id': 'req-ea582629-19f6-4cfb-8f91-d501f5f8f0dc'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:21.190568 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-ea582629-19f6-4cfb-8f91-d501f5f8f0dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dec0d78dbb3494983bfead5385ab10f', 'request_id': 'req-ea582629-19f6-4cfb-8f91-d501f5f8f0dc'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:21.284479 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-ea582629-19f6-4cfb-8f91-d501f5f8f0dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dec0d78dbb3494983bfead5385ab10f', 'request_id': 'req-ea582629-19f6-4cfb-8f91-d501f5f8f0dc'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:21.295147 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-ea582629-19f6-4cfb-8f91-d501f5f8f0dc admin admin] The token cache is being invalidate because project 8dec0d78dbb3494983bfead5385ab10f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:43:21.296208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-ea582629-19f6-4cfb-8f91-d501f5f8f0dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:21.296208 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-ea582629-19f6-4cfb-8f91-d501f5f8f0dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:21.297699 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1369/2757] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:43:21 2022] DELETE /identity/v3/projects/8dec0d78dbb3494983bfead5385ab10f => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:43:21.302578 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae82f1-4d81-4233-8f9e-e5ee4f70b0fc None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:21.303048 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae82f1-4d81-4233-8f9e-e5ee4f70b0fc None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:21.303176 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae82f1-4d81-4233-8f9e-e5ee4f70b0fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:21.322404 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-b8ae82f1-4d81-4233-8f9e-e5ee4f70b0fc None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:43:21.323611 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-b8ae82f1-4d81-4233-8f9e-e5ee4f70b0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:21.353344 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:43:21.382467 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1389/2758] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:43:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:43:21.390815 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140cb450-69ab-4ec3-afc6-2f492361e3e3 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:21.394062 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140cb450-69ab-4ec3-afc6-2f492361e3e3 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:21.394062 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140cb450-69ab-4ec3-afc6-2f492361e3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:21.415973 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-140cb450-69ab-4ec3-afc6-2f492361e3e3 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:43:21.426622 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-140cb450-69ab-4ec3-afc6-2f492361e3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:21.440953 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1370/2759] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:43:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:43:21.445128 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae82f1-4d81-4233-8f9e-e5ee4f70b0fc None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:21.455387 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae82f1-4d81-4233-8f9e-e5ee4f70b0fc None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:21.460037 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e1ff1-d647-4dc3-be9b-0b112693d080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:21.460673 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e1ff1-d647-4dc3-be9b-0b112693d080 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:21.460962 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e1ff1-d647-4dc3-be9b-0b112693d080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:21.461239 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e1ff1-d647-4dc3-be9b-0b112693d080 admin admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:21.461933 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702e1ff1-d647-4dc3-be9b-0b112693d080 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:43:21.462242 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702e1ff1-d647-4dc3-be9b-0b112693d080 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:21.463827 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702e1ff1-d647-4dc3-be9b-0b112693d080 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:21.476664 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1390/2760] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:43:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:43:21.480945 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140cb450-69ab-4ec3-afc6-2f492361e3e3 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:21.490163 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140cb450-69ab-4ec3-afc6-2f492361e3e3 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:21.494931 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29159563-b104-4ffa-93ae-b05549f5ee92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:21.495537 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29159563-b104-4ffa-93ae-b05549f5ee92 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:21.495823 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29159563-b104-4ffa-93ae-b05549f5ee92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:21.496132 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29159563-b104-4ffa-93ae-b05549f5ee92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:21.496805 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29159563-b104-4ffa-93ae-b05549f5ee92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:21.499246 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29159563-b104-4ffa-93ae-b05549f5ee92 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:21.574942 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1371/2761] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:43:21 2022] POST /identity/v3/projects => generated 389 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:43:21.579623 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e1ff1-d647-4dc3-be9b-0b112693d080 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:21.589786 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e1ff1-d647-4dc3-be9b-0b112693d080 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:21.645946 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68924138-aa47-4b51-968d-1ab8912f9f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:21.646485 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68924138-aa47-4b51-968d-1ab8912f9f24 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:21.646770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68924138-aa47-4b51-968d-1ab8912f9f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:21.647056 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68924138-aa47-4b51-968d-1ab8912f9f24 admin admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:21.647714 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68924138-aa47-4b51-968d-1ab8912f9f24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:21.649673 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68924138-aa47-4b51-968d-1ab8912f9f24 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:21.718545 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1391/2762] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:43:21 2022] POST /identity/v3/users => generated 295 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:43:21.722671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29159563-b104-4ffa-93ae-b05549f5ee92 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:21.733238 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29159563-b104-4ffa-93ae-b05549f5ee92 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:21.737902 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f88c03-3baa-44f4-aa72-c99dc5a08abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:21.738508 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f88c03-3baa-44f4-aa72-c99dc5a08abf admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:21.738823 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f88c03-3baa-44f4-aa72-c99dc5a08abf admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:21.739255 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f88c03-3baa-44f4-aa72-c99dc5a08abf admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:21.739976 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f88c03-3baa-44f4-aa72-c99dc5a08abf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:21.741605 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f88c03-3baa-44f4-aa72-c99dc5a08abf admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:21.756374 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1372/2763] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:43:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:43:21.760628 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68924138-aa47-4b51-968d-1ab8912f9f24 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:21.774221 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68924138-aa47-4b51-968d-1ab8912f9f24 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:21.787817 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5cc09a-e874-4659-b7bb-20f9d409ffaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:21.789196 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5cc09a-e874-4659-b7bb-20f9d409ffaf admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:21.789545 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5cc09a-e874-4659-b7bb-20f9d409ffaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:21.789823 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5cc09a-e874-4659-b7bb-20f9d409ffaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:21.790946 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5cc09a-e874-4659-b7bb-20f9d409ffaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:21.794261 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5cc09a-e874-4659-b7bb-20f9d409ffaf admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:21.806226 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1392/2764] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:43:21 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 15:43:21.810886 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f88c03-3baa-44f4-aa72-c99dc5a08abf admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:21.820854 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f88c03-3baa-44f4-aa72-c99dc5a08abf admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:21.826038 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468100dc-817e-488d-b5b6-7b85df26dc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:21.826683 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468100dc-817e-488d-b5b6-7b85df26dc04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:21.826968 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468100dc-817e-488d-b5b6-7b85df26dc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:21.827240 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468100dc-817e-488d-b5b6-7b85df26dc04 admin admin] PATH_INFO: `/v3/projects/d6155bfb9f9147d68d82d247261caa77/users/caeb0d33737e44ea85ec171b315a0889/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:21.841469 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468100dc-817e-488d-b5b6-7b85df26dc04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6155bfb9f9147d68d82d247261caa77, user_id=caeb0d33737e44ea85ec171b315a0889, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:21.844093 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468100dc-817e-488d-b5b6-7b85df26dc04 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:21.933497 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1373/2765] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:43:21 2022] PUT /identity/v3/projects/d6155bfb9f9147d68d82d247261caa77/users/caeb0d33737e44ea85ec171b315a0889/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:43:21.945620 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb85db-a1e4-4379-9f90-2d2c8ce23d3a None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:21.949118 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb85db-a1e4-4379-9f90-2d2c8ce23d3a None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:21.949118 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb85db-a1e4-4379-9f90-2d2c8ce23d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:21.973145 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-fbbb85db-a1e4-4379-9f90-2d2c8ce23d3a None None] MFA Rules not processed for user `caeb0d33737e44ea85ec171b315a0889`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:43:21.974358 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-fbbb85db-a1e4-4379-9f90-2d2c8ce23d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:22.036692 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1393/2766] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:43:21 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:43:22.041481 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468100dc-817e-488d-b5b6-7b85df26dc04 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:22.051044 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:43:22.116539 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468100dc-817e-488d-b5b6-7b85df26dc04 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:22.121324 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b77aed-3374-4a1a-b731-a7eb0b1fceaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:22.121942 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b77aed-3374-4a1a-b731-a7eb0b1fceaf admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:22.122229 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b77aed-3374-4a1a-b731-a7eb0b1fceaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:22.123156 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b77aed-3374-4a1a-b731-a7eb0b1fceaf admin admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:22.123825 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b77aed-3374-4a1a-b731-a7eb0b1fceaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:22.125763 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b77aed-3374-4a1a-b731-a7eb0b1fceaf admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:22.209608 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1374/2767] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:43:22 2022] POST /identity/v3/projects => generated 389 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:43:22.213586 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb85db-a1e4-4379-9f90-2d2c8ce23d3a None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:22.224267 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb85db-a1e4-4379-9f90-2d2c8ce23d3a None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:22.251775 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:43:22.281985 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730f8bf-b44c-4602-b466-0f301906deba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:22.282945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730f8bf-b44c-4602-b466-0f301906deba admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:22.283431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730f8bf-b44c-4602-b466-0f301906deba admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:22.283875 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730f8bf-b44c-4602-b466-0f301906deba admin admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:22.284934 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730f8bf-b44c-4602-b466-0f301906deba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:22.286923 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730f8bf-b44c-4602-b466-0f301906deba admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:22.369860 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1394/2768] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:43:22 2022] POST /identity/v3/users => generated 301 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:43:22.377556 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b77aed-3374-4a1a-b731-a7eb0b1fceaf admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:22.390066 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b77aed-3374-4a1a-b731-a7eb0b1fceaf admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:22.394990 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f902e210-cfe5-4911-a1cc-473e6dbb4e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:22.395605 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902e210-cfe5-4911-a1cc-473e6dbb4e22 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:22.395987 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902e210-cfe5-4911-a1cc-473e6dbb4e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:22.396294 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902e210-cfe5-4911-a1cc-473e6dbb4e22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:22.396980 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f902e210-cfe5-4911-a1cc-473e6dbb4e22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:22.399160 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f902e210-cfe5-4911-a1cc-473e6dbb4e22 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:22.410909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1375/2769] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:43:22.415156 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730f8bf-b44c-4602-b466-0f301906deba admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:22.426346 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730f8bf-b44c-4602-b466-0f301906deba admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:22.430980 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a8794-4538-41f7-9d55-600ea5782338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:22.431637 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a8794-4538-41f7-9d55-600ea5782338 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:22.431923 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a8794-4538-41f7-9d55-600ea5782338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:22.432235 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a8794-4538-41f7-9d55-600ea5782338 admin admin] PATH_INFO: `/v3/projects/c1de9b3504f94ef680525d504fa85e5e/users/332205e193144e5ab6611b9d47054ef4/roles/7c88c815db854954b516ab97cd471647` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:22.446927 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404a8794-4538-41f7-9d55-600ea5782338 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1de9b3504f94ef680525d504fa85e5e, user_id=332205e193144e5ab6611b9d47054ef4, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:22.449673 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404a8794-4538-41f7-9d55-600ea5782338 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:22.506569 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1395/2770] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:43:22 2022] PUT /identity/v3/projects/c1de9b3504f94ef680525d504fa85e5e/users/332205e193144e5ab6611b9d47054ef4/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:43:22.510429 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f902e210-cfe5-4911-a1cc-473e6dbb4e22 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:22.520451 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f902e210-cfe5-4911-a1cc-473e6dbb4e22 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:22.576345 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c81adf-545d-4789-a0b7-a01678f856f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:22.576953 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c81adf-545d-4789-a0b7-a01678f856f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:22.577237 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c81adf-545d-4789-a0b7-a01678f856f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:22.577511 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c81adf-545d-4789-a0b7-a01678f856f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:22.578192 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c81adf-545d-4789-a0b7-a01678f856f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:22.579766 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c81adf-545d-4789-a0b7-a01678f856f2 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:22.589216 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1376/2771] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:43:22.593803 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a8794-4538-41f7-9d55-600ea5782338 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:22.604159 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a8794-4538-41f7-9d55-600ea5782338 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:22.609519 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497cc3ad-637d-4a96-a3d1-3bc68e8fd798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:22.610177 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497cc3ad-637d-4a96-a3d1-3bc68e8fd798 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:22.610464 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497cc3ad-637d-4a96-a3d1-3bc68e8fd798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:22.610743 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497cc3ad-637d-4a96-a3d1-3bc68e8fd798 admin admin] PATH_INFO: `/v3/projects/c1de9b3504f94ef680525d504fa85e5e/users/332205e193144e5ab6611b9d47054ef4/roles/7c88c815db854954b516ab97cd471647` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:22.612736 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497cc3ad-637d-4a96-a3d1-3bc68e8fd798 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1de9b3504f94ef680525d504fa85e5e, user_id=332205e193144e5ab6611b9d47054ef4, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:22.615351 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497cc3ad-637d-4a96-a3d1-3bc68e8fd798 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:22.625762 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1396/2772] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:43:22 2022] PUT /identity/v3/projects/c1de9b3504f94ef680525d504fa85e5e/users/332205e193144e5ab6611b9d47054ef4/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:43:22.635640 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac0e2e-e6f5-49f5-a782-b1210652b43d None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:22.635929 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac0e2e-e6f5-49f5-a782-b1210652b43d None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:22.636242 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac0e2e-e6f5-49f5-a782-b1210652b43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:22.664494 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-46ac0e2e-e6f5-49f5-a782-b1210652b43d None None] MFA Rules not processed for user `332205e193144e5ab6611b9d47054ef4`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:43:22.670757 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-46ac0e2e-e6f5-49f5-a782-b1210652b43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:22.739355 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1377/2773] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:43:22 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:43:22.750340 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497cc3ad-637d-4a96-a3d1-3bc68e8fd798 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:22.752047 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-497cc3ad-637d-4a96-a3d1-3bc68e8fd798 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:22.809314 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497cc3ad-637d-4a96-a3d1-3bc68e8fd798 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:22.812935 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd108fd-6c31-40d0-9e57-08ec636b2795 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:22.813546 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd108fd-6c31-40d0-9e57-08ec636b2795 service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:22.813833 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd108fd-6c31-40d0-9e57-08ec636b2795 service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:22.814112 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd108fd-6c31-40d0-9e57-08ec636b2795 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:22.822974 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd108fd-6c31-40d0-9e57-08ec636b2795 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:22.824449 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd108fd-6c31-40d0-9e57-08ec636b2795 service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:22.834272 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1397/2774] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:43:22 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:43:22.968852 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac0e2e-e6f5-49f5-a782-b1210652b43d None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:22.973164 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:43:23.031881 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac0e2e-e6f5-49f5-a782-b1210652b43d None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:23.036925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971d07df-f4c5-4586-812b-95f8c8d6a94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:23.037562 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d07df-f4c5-4586-812b-95f8c8d6a94c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:23.037844 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d07df-f4c5-4586-812b-95f8c8d6a94c admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:23.038119 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d07df-f4c5-4586-812b-95f8c8d6a94c admin admin] PATH_INFO: `/v3/users/caeb0d33737e44ea85ec171b315a0889` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:23.039269 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971d07df-f4c5-4586-812b-95f8c8d6a94c admin admin] RBAC: Authorizing `identity:delete_user(user_id=caeb0d33737e44ea85ec171b315a0889)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:23.041206 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971d07df-f4c5-4586-812b-95f8c8d6a94c admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:23.200423 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-971d07df-f4c5-4586-812b-95f8c8d6a94c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'caeb0d33737e44ea85ec171b315a0889'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:23.207918 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-971d07df-f4c5-4586-812b-95f8c8d6a94c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'caeb0d33737e44ea85ec171b315a0889'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:23.208623 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-971d07df-f4c5-4586-812b-95f8c8d6a94c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'caeb0d33737e44ea85ec171b315a0889'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:23.220378 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-971d07df-f4c5-4586-812b-95f8c8d6a94c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'caeb0d33737e44ea85ec171b315a0889'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:23.221018 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-971d07df-f4c5-4586-812b-95f8c8d6a94c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'caeb0d33737e44ea85ec171b315a0889'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:23.284282 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1378/2775] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:43:22 2022] DELETE /identity/v3/users/caeb0d33737e44ea85ec171b315a0889 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:43:23.288822 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032bd0ec-8bf6-482e-bfa0-18eca13a9a20 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:23.289117 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032bd0ec-8bf6-482e-bfa0-18eca13a9a20 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:23.289399 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032bd0ec-8bf6-482e-bfa0-18eca13a9a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:23.307704 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-032bd0ec-8bf6-482e-bfa0-18eca13a9a20 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:43:23.308961 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-032bd0ec-8bf6-482e-bfa0-18eca13a9a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:23.367353 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1398/2776] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:43:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:43:23.377484 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971d07df-f4c5-4586-812b-95f8c8d6a94c admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:23.378865 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-971d07df-f4c5-4586-812b-95f8c8d6a94c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:23.437005 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971d07df-f4c5-4586-812b-95f8c8d6a94c admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:23.440601 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8aa31a-0859-4127-9c1a-aa561f9a8e0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:23.441209 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8aa31a-0859-4127-9c1a-aa561f9a8e0f service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:23.441496 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8aa31a-0859-4127-9c1a-aa561f9a8e0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:23.441776 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8aa31a-0859-4127-9c1a-aa561f9a8e0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:23.450838 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8aa31a-0859-4127-9c1a-aa561f9a8e0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:23.452290 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8aa31a-0859-4127-9c1a-aa561f9a8e0f service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:23.461729 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1379/2777] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:43:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 15:43:23.840232 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032bd0ec-8bf6-482e-bfa0-18eca13a9a20 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:23.843002 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-032bd0ec-8bf6-482e-bfa0-18eca13a9a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:23.854334 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032bd0ec-8bf6-482e-bfa0-18eca13a9a20 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:23.859711 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53f9b16-8251-4e06-98b3-34fbd4136fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:23.860468 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f9b16-8251-4e06-98b3-34fbd4136fbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:23.860748 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f9b16-8251-4e06-98b3-34fbd4136fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:23.861026 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f9b16-8251-4e06-98b3-34fbd4136fbc admin admin] PATH_INFO: `/v3/projects/d6155bfb9f9147d68d82d247261caa77` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:23.862178 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53f9b16-8251-4e06-98b3-34fbd4136fbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6155bfb9f9147d68d82d247261caa77)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:23.864088 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53f9b16-8251-4e06-98b3-34fbd4136fbc admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:23.885089 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:43:23.951683 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-e53f9b16-8251-4e06-98b3-34fbd4136fbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6155bfb9f9147d68d82d247261caa77', 'request_id': 'req-e53f9b16-8251-4e06-98b3-34fbd4136fbc'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:23.957976 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-e53f9b16-8251-4e06-98b3-34fbd4136fbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6155bfb9f9147d68d82d247261caa77', 'request_id': 'req-e53f9b16-8251-4e06-98b3-34fbd4136fbc'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:23.963729 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-e53f9b16-8251-4e06-98b3-34fbd4136fbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6155bfb9f9147d68d82d247261caa77', 'request_id': 'req-e53f9b16-8251-4e06-98b3-34fbd4136fbc'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:24.036753 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-e53f9b16-8251-4e06-98b3-34fbd4136fbc admin admin] The token cache is being invalidate because project d6155bfb9f9147d68d82d247261caa77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:43:24.037066 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-e53f9b16-8251-4e06-98b3-34fbd4136fbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:24.037681 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-e53f9b16-8251-4e06-98b3-34fbd4136fbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:24.039199 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1399/2778] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:43:23 2022] DELETE /identity/v3/projects/d6155bfb9f9147d68d82d247261caa77 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:43:24.043145 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8aa31a-0859-4127-9c1a-aa561f9a8e0f service neutron] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:24.053800 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8aa31a-0859-4127-9c1a-aa561f9a8e0f service neutron] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:24.073863 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:43:24.109598 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb545a4c-6a53-4178-9bda-c6fca495360f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:24.110230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb545a4c-6a53-4178-9bda-c6fca495360f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:24.110513 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb545a4c-6a53-4178-9bda-c6fca495360f admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:24.110786 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb545a4c-6a53-4178-9bda-c6fca495360f admin admin] PATH_INFO: `/v3/users/332205e193144e5ab6611b9d47054ef4` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:24.111937 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb545a4c-6a53-4178-9bda-c6fca495360f admin admin] RBAC: Authorizing `identity:delete_user(user_id=332205e193144e5ab6611b9d47054ef4)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:24.113869 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb545a4c-6a53-4178-9bda-c6fca495360f admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:24.260755 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-eb545a4c-6a53-4178-9bda-c6fca495360f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '332205e193144e5ab6611b9d47054ef4'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:24.268010 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-eb545a4c-6a53-4178-9bda-c6fca495360f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '332205e193144e5ab6611b9d47054ef4'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:24.268701 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-eb545a4c-6a53-4178-9bda-c6fca495360f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '332205e193144e5ab6611b9d47054ef4'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:24.281134 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-eb545a4c-6a53-4178-9bda-c6fca495360f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '332205e193144e5ab6611b9d47054ef4'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:24.281860 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-eb545a4c-6a53-4178-9bda-c6fca495360f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '332205e193144e5ab6611b9d47054ef4'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:24.381549 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1380/2779] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:43:24 2022] DELETE /identity/v3/users/332205e193144e5ab6611b9d47054ef4 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:43:24.786703 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53f9b16-8251-4e06-98b3-34fbd4136fbc admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:24.788213 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-e53f9b16-8251-4e06-98b3-34fbd4136fbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:24.801567 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53f9b16-8251-4e06-98b3-34fbd4136fbc admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:24.862439 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a817cce8-f818-41b4-b699-4a177dc66bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:24.863041 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817cce8-f818-41b4-b699-4a177dc66bc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:24.863281 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817cce8-f818-41b4-b699-4a177dc66bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:24.863564 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817cce8-f818-41b4-b699-4a177dc66bc1 admin admin] PATH_INFO: `/v3/projects/c1de9b3504f94ef680525d504fa85e5e` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:24.864729 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a817cce8-f818-41b4-b699-4a177dc66bc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1de9b3504f94ef680525d504fa85e5e)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:24.866609 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a817cce8-f818-41b4-b699-4a177dc66bc1 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:24.962054 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-a817cce8-f818-41b4-b699-4a177dc66bc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1de9b3504f94ef680525d504fa85e5e', 'request_id': 'req-a817cce8-f818-41b4-b699-4a177dc66bc1'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:24.968159 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-a817cce8-f818-41b4-b699-4a177dc66bc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1de9b3504f94ef680525d504fa85e5e', 'request_id': 'req-a817cce8-f818-41b4-b699-4a177dc66bc1'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:24.973763 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-a817cce8-f818-41b4-b699-4a177dc66bc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1de9b3504f94ef680525d504fa85e5e', 'request_id': 'req-a817cce8-f818-41b4-b699-4a177dc66bc1'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:25.043494 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-a817cce8-f818-41b4-b699-4a177dc66bc1 admin admin] The token cache is being invalidate because project c1de9b3504f94ef680525d504fa85e5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:43:25.043823 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-a817cce8-f818-41b4-b699-4a177dc66bc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:25.045084 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-a817cce8-f818-41b4-b699-4a177dc66bc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:25.046539 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1400/2780] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:43:24 2022] DELETE /identity/v3/projects/c1de9b3504f94ef680525d504fa85e5e => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:43:25.051422 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f03b95-3f2f-4bba-83d6-18223f99dbf3 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:25.051720 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f03b95-3f2f-4bba-83d6-18223f99dbf3 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:25.052034 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f03b95-3f2f-4bba-83d6-18223f99dbf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:25.072571 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-80f03b95-3f2f-4bba-83d6-18223f99dbf3 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:43:25.073789 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-80f03b95-3f2f-4bba-83d6-18223f99dbf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:25.132240 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1381/2781] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:43:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:43:25.140258 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38deb5cb-90b3-4798-a99d-f12b2b1187e4 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:25.140553 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38deb5cb-90b3-4798-a99d-f12b2b1187e4 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:25.140839 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38deb5cb-90b3-4798-a99d-f12b2b1187e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:25.161817 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-38deb5cb-90b3-4798-a99d-f12b2b1187e4 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:43:25.163017 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-38deb5cb-90b3-4798-a99d-f12b2b1187e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:25.171723 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1401/2782] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:43:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:43:25.175877 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f03b95-3f2f-4bba-83d6-18223f99dbf3 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:25.186317 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f03b95-3f2f-4bba-83d6-18223f99dbf3 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:25.190823 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c1359b-9064-4137-81f3-8ff62a771ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:25.191427 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1359b-9064-4137-81f3-8ff62a771ec4 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:25.191712 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1359b-9064-4137-81f3-8ff62a771ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:25.191991 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1359b-9064-4137-81f3-8ff62a771ec4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:25.192709 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c1359b-9064-4137-81f3-8ff62a771ec4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:43:25.193020 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c1359b-9064-4137-81f3-8ff62a771ec4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:25.194616 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c1359b-9064-4137-81f3-8ff62a771ec4 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:25.205996 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1382/2783] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:43:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:43:25.210214 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38deb5cb-90b3-4798-a99d-f12b2b1187e4 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:25.219359 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38deb5cb-90b3-4798-a99d-f12b2b1187e4 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:25.223920 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985dccf6-ccbf-43ab-8c8f-8d82075c9b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:25.224555 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985dccf6-ccbf-43ab-8c8f-8d82075c9b8e admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:25.224836 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985dccf6-ccbf-43ab-8c8f-8d82075c9b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:25.225110 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985dccf6-ccbf-43ab-8c8f-8d82075c9b8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:25.225774 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985dccf6-ccbf-43ab-8c8f-8d82075c9b8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:25.227687 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985dccf6-ccbf-43ab-8c8f-8d82075c9b8e admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:25.314660 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1402/2784] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:43:25 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:43:25.320469 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c1359b-9064-4137-81f3-8ff62a771ec4 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:25.332821 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c1359b-9064-4137-81f3-8ff62a771ec4 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:25.362112 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:43:25.389656 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbce8038-735c-4506-9bfb-15ba6a0a0753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:25.390269 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce8038-735c-4506-9bfb-15ba6a0a0753 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:25.390554 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce8038-735c-4506-9bfb-15ba6a0a0753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:25.390833 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce8038-735c-4506-9bfb-15ba6a0a0753 admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:25.391486 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbce8038-735c-4506-9bfb-15ba6a0a0753 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:25.393425 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbce8038-735c-4506-9bfb-15ba6a0a0753 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:25.489567 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1383/2785] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:43:25 2022] POST /identity/v3/users => generated 294 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:43:25.493659 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985dccf6-ccbf-43ab-8c8f-8d82075c9b8e admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:25.504034 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985dccf6-ccbf-43ab-8c8f-8d82075c9b8e admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:25.508822 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef47575b-ba3a-491f-8f0b-93c1bf11d181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:25.510713 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef47575b-ba3a-491f-8f0b-93c1bf11d181 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:25.510713 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef47575b-ba3a-491f-8f0b-93c1bf11d181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:25.510713 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef47575b-ba3a-491f-8f0b-93c1bf11d181 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:25.510713 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef47575b-ba3a-491f-8f0b-93c1bf11d181 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:25.512285 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef47575b-ba3a-491f-8f0b-93c1bf11d181 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:25.514326 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:43:25.523149 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1403/2786] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:43:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:43:25.526898 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbce8038-735c-4506-9bfb-15ba6a0a0753 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:25.537655 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbce8038-735c-4506-9bfb-15ba6a0a0753 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:25.542300 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7607aa36-4895-4689-9e8b-945ae16e229c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:25.542900 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607aa36-4895-4689-9e8b-945ae16e229c admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:25.543185 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607aa36-4895-4689-9e8b-945ae16e229c admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:25.543464 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607aa36-4895-4689-9e8b-945ae16e229c admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:25.544188 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7607aa36-4895-4689-9e8b-945ae16e229c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:25.546359 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7607aa36-4895-4689-9e8b-945ae16e229c admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:25.556874 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1384/2787] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:43:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:43:25.560947 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef47575b-ba3a-491f-8f0b-93c1bf11d181 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:25.570795 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef47575b-ba3a-491f-8f0b-93c1bf11d181 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:25.575544 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c9b114-99e1-4dca-b7b5-b58e0b9014c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:25.576259 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9b114-99e1-4dca-b7b5-b58e0b9014c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:25.576895 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9b114-99e1-4dca-b7b5-b58e0b9014c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:25.577231 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9b114-99e1-4dca-b7b5-b58e0b9014c8 admin admin] PATH_INFO: `/v3/projects/0a8684d9ff7f449295364dff438d34d5/users/cd9b75b1ef4a47c0bfe44b0ef49ab7e1/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:25.591810 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c9b114-99e1-4dca-b7b5-b58e0b9014c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a8684d9ff7f449295364dff438d34d5, user_id=cd9b75b1ef4a47c0bfe44b0ef49ab7e1, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:25.594478 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c9b114-99e1-4dca-b7b5-b58e0b9014c8 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:25.644281 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1404/2788] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:43:25 2022] PUT /identity/v3/projects/0a8684d9ff7f449295364dff438d34d5/users/cd9b75b1ef4a47c0bfe44b0ef49ab7e1/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:43:25.710774 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50376218-acd0-4a8c-804c-9ad358b6b28e None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:25.711064 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50376218-acd0-4a8c-804c-9ad358b6b28e None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:25.711348 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50376218-acd0-4a8c-804c-9ad358b6b28e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:25.739165 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-50376218-acd0-4a8c-804c-9ad358b6b28e None None] MFA Rules not processed for user `cd9b75b1ef4a47c0bfe44b0ef49ab7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:43:25.740438 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-50376218-acd0-4a8c-804c-9ad358b6b28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:25.802811 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1385/2789] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:43:25 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:43:25.807301 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c9b114-99e1-4dca-b7b5-b58e0b9014c8 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:25.868241 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c9b114-99e1-4dca-b7b5-b58e0b9014c8 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:25.872923 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6118ab0c-90d6-4725-b508-ffd4e631be73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:25.873532 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6118ab0c-90d6-4725-b508-ffd4e631be73 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:25.873818 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6118ab0c-90d6-4725-b508-ffd4e631be73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:25.874099 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6118ab0c-90d6-4725-b508-ffd4e631be73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:25.874766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6118ab0c-90d6-4725-b508-ffd4e631be73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:25.876717 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6118ab0c-90d6-4725-b508-ffd4e631be73 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:25.953971 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1405/2790] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:43:25 2022] POST /identity/v3/projects => generated 385 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:43:25.958221 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50376218-acd0-4a8c-804c-9ad358b6b28e None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:25.969941 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50376218-acd0-4a8c-804c-9ad358b6b28e None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:26.025772 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a297a7-1f8e-4ec5-aabe-15add5b956f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:26.026394 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a297a7-1f8e-4ec5-aabe-15add5b956f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:26.026679 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a297a7-1f8e-4ec5-aabe-15add5b956f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:26.026955 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a297a7-1f8e-4ec5-aabe-15add5b956f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:26.027609 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a297a7-1f8e-4ec5-aabe-15add5b956f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:26.029621 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a297a7-1f8e-4ec5-aabe-15add5b956f0 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:26.128632 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1386/2791] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:43:25 2022] POST /identity/v3/users => generated 299 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:43:26.132526 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6118ab0c-90d6-4725-b508-ffd4e631be73 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:26.143532 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6118ab0c-90d6-4725-b508-ffd4e631be73 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:26.148306 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4690567-459f-4462-b8e3-e1ffe6786658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:26.148916 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4690567-459f-4462-b8e3-e1ffe6786658 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:26.149197 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4690567-459f-4462-b8e3-e1ffe6786658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:26.149578 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4690567-459f-4462-b8e3-e1ffe6786658 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:26.150297 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4690567-459f-4462-b8e3-e1ffe6786658 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:26.151879 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4690567-459f-4462-b8e3-e1ffe6786658 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:26.162702 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1406/2792] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:43:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:43:26.166980 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a297a7-1f8e-4ec5-aabe-15add5b956f0 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:26.177622 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a297a7-1f8e-4ec5-aabe-15add5b956f0 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:26.182504 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50898ce2-906e-4812-a92d-59a23d1d9197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:26.183172 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50898ce2-906e-4812-a92d-59a23d1d9197 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:26.183489 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50898ce2-906e-4812-a92d-59a23d1d9197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:26.183775 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50898ce2-906e-4812-a92d-59a23d1d9197 admin admin] PATH_INFO: `/v3/projects/c7b2fe0310f0422c8922eefa3c88cf6f/users/4294003b410b4b5da4b8d0b4aa839593/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:26.199043 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50898ce2-906e-4812-a92d-59a23d1d9197 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7b2fe0310f0422c8922eefa3c88cf6f, user_id=4294003b410b4b5da4b8d0b4aa839593, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:26.202393 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50898ce2-906e-4812-a92d-59a23d1d9197 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:26.263227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1387/2793] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:43:26 2022] PUT /identity/v3/projects/c7b2fe0310f0422c8922eefa3c88cf6f/users/4294003b410b4b5da4b8d0b4aa839593/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:43:26.267967 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4690567-459f-4462-b8e3-e1ffe6786658 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:26.279500 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4690567-459f-4462-b8e3-e1ffe6786658 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:26.343906 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13089fef-c9b9-489a-9bd9-6ec30c9df6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:26.344565 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13089fef-c9b9-489a-9bd9-6ec30c9df6d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:26.344855 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13089fef-c9b9-489a-9bd9-6ec30c9df6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:26.345135 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13089fef-c9b9-489a-9bd9-6ec30c9df6d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:26.345836 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13089fef-c9b9-489a-9bd9-6ec30c9df6d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:26.347431 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13089fef-c9b9-489a-9bd9-6ec30c9df6d8 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:26.356950 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1407/2794] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:43:26 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:43:26.361647 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50898ce2-906e-4812-a92d-59a23d1d9197 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:26.368263 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:43:26.371957 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50898ce2-906e-4812-a92d-59a23d1d9197 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:26.377226 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af0418-e34c-4a68-be04-35f1129a85f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:26.383965 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af0418-e34c-4a68-be04-35f1129a85f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:26.383965 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af0418-e34c-4a68-be04-35f1129a85f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:26.383965 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af0418-e34c-4a68-be04-35f1129a85f4 admin admin] PATH_INFO: `/v3/projects/c7b2fe0310f0422c8922eefa3c88cf6f/users/4294003b410b4b5da4b8d0b4aa839593/roles/7c88c815db854954b516ab97cd471647` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:26.383965 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af0418-e34c-4a68-be04-35f1129a85f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7b2fe0310f0422c8922eefa3c88cf6f, user_id=4294003b410b4b5da4b8d0b4aa839593, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:26.383965 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af0418-e34c-4a68-be04-35f1129a85f4 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:26.393310 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1388/2795] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:43:26 2022] PUT /identity/v3/projects/c7b2fe0310f0422c8922eefa3c88cf6f/users/4294003b410b4b5da4b8d0b4aa839593/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:43:26.403224 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc50975f-1c6e-4e80-a54e-8d77bcf22adb None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:26.403521 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc50975f-1c6e-4e80-a54e-8d77bcf22adb None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:26.403834 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc50975f-1c6e-4e80-a54e-8d77bcf22adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:26.431780 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-fc50975f-1c6e-4e80-a54e-8d77bcf22adb None None] MFA Rules not processed for user `4294003b410b4b5da4b8d0b4aa839593`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:43:26.433036 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-fc50975f-1c6e-4e80-a54e-8d77bcf22adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:26.465703 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:43:26.501623 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1408/2796] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:43:26 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:43:26.512343 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af0418-e34c-4a68-be04-35f1129a85f4 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:26.514123 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-69af0418-e34c-4a68-be04-35f1129a85f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:26.571790 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af0418-e34c-4a68-be04-35f1129a85f4 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:26.575477 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45a21e-14db-48d0-ae95-a554ea81c1fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:26.576138 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45a21e-14db-48d0-ae95-a554ea81c1fd service cinder] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:26.576428 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45a21e-14db-48d0-ae95-a554ea81c1fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:26.576702 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45a21e-14db-48d0-ae95-a554ea81c1fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:26.589678 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f45a21e-14db-48d0-ae95-a554ea81c1fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:26.591221 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f45a21e-14db-48d0-ae95-a554ea81c1fd service cinder] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:26.602827 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1389/2797] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:43:26 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:43:27.622004 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc50975f-1c6e-4e80-a54e-8d77bcf22adb None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:27.685218 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc50975f-1c6e-4e80-a54e-8d77bcf22adb None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:27.690318 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b0046a-2a24-4580-a99b-7519f2ce4389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:27.690952 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0046a-2a24-4580-a99b-7519f2ce4389 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:27.691233 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0046a-2a24-4580-a99b-7519f2ce4389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:27.691510 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0046a-2a24-4580-a99b-7519f2ce4389 admin admin] PATH_INFO: `/v3/users/cd9b75b1ef4a47c0bfe44b0ef49ab7e1` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:27.692747 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b0046a-2a24-4580-a99b-7519f2ce4389 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd9b75b1ef4a47c0bfe44b0ef49ab7e1)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:27.694647 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b0046a-2a24-4580-a99b-7519f2ce4389 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:27.857348 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-a6b0046a-2a24-4580-a99b-7519f2ce4389 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd9b75b1ef4a47c0bfe44b0ef49ab7e1'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:27.858488 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-a6b0046a-2a24-4580-a99b-7519f2ce4389 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd9b75b1ef4a47c0bfe44b0ef49ab7e1'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:27.870193 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-a6b0046a-2a24-4580-a99b-7519f2ce4389 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd9b75b1ef4a47c0bfe44b0ef49ab7e1'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:27.929786 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-a6b0046a-2a24-4580-a99b-7519f2ce4389 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd9b75b1ef4a47c0bfe44b0ef49ab7e1'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:27.937386 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-a6b0046a-2a24-4580-a99b-7519f2ce4389 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd9b75b1ef4a47c0bfe44b0ef49ab7e1'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:27.943259 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1409/2798] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:43:27 2022] DELETE /identity/v3/users/cd9b75b1ef4a47c0bfe44b0ef49ab7e1 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:43:27.947708 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e14699-607d-496a-aca2-3af488e7a97f None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:27.948210 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e14699-607d-496a-aca2-3af488e7a97f None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:27.948327 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e14699-607d-496a-aca2-3af488e7a97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:27.966623 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-09e14699-607d-496a-aca2-3af488e7a97f None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:43:27.967831 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-09e14699-607d-496a-aca2-3af488e7a97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:28.027049 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1390/2799] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:43:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:43:28.036818 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b0046a-2a24-4580-a99b-7519f2ce4389 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:28.038241 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-a6b0046a-2a24-4580-a99b-7519f2ce4389 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:28.091261 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:43:28.095076 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b0046a-2a24-4580-a99b-7519f2ce4389 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:28.098751 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f261681-fc32-4f4f-8e08-c467ff0d6e94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:28.099370 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f261681-fc32-4f4f-8e08-c467ff0d6e94 service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:28.099657 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f261681-fc32-4f4f-8e08-c467ff0d6e94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:28.099993 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f261681-fc32-4f4f-8e08-c467ff0d6e94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:28.111464 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f261681-fc32-4f4f-8e08-c467ff0d6e94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:28.111464 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f261681-fc32-4f4f-8e08-c467ff0d6e94 service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:28.120290 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1410/2800] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:43:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:43:28.537742 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e14699-607d-496a-aca2-3af488e7a97f None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:28.540852 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-09e14699-607d-496a-aca2-3af488e7a97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:28.552262 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e14699-607d-496a-aca2-3af488e7a97f None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:28.557131 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c92a97-4589-4802-8804-13679c3b781d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:28.557787 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c92a97-4589-4802-8804-13679c3b781d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:28.558071 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c92a97-4589-4802-8804-13679c3b781d admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:28.558352 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c92a97-4589-4802-8804-13679c3b781d admin admin] PATH_INFO: `/v3/projects/0a8684d9ff7f449295364dff438d34d5` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:28.559520 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c92a97-4589-4802-8804-13679c3b781d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a8684d9ff7f449295364dff438d34d5)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:28.561512 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c92a97-4589-4802-8804-13679c3b781d admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:28.679546 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-d4c92a97-4589-4802-8804-13679c3b781d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a8684d9ff7f449295364dff438d34d5', 'request_id': 'req-d4c92a97-4589-4802-8804-13679c3b781d'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:28.685874 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-d4c92a97-4589-4802-8804-13679c3b781d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a8684d9ff7f449295364dff438d34d5', 'request_id': 'req-d4c92a97-4589-4802-8804-13679c3b781d'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:28.739100 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-d4c92a97-4589-4802-8804-13679c3b781d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a8684d9ff7f449295364dff438d34d5', 'request_id': 'req-d4c92a97-4589-4802-8804-13679c3b781d'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:28.749799 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-d4c92a97-4589-4802-8804-13679c3b781d admin admin] The token cache is being invalidate because project 0a8684d9ff7f449295364dff438d34d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:43:28.750088 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-d4c92a97-4589-4802-8804-13679c3b781d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:28.750710 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-d4c92a97-4589-4802-8804-13679c3b781d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:28.752248 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1391/2801] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:43:28 2022] DELETE /identity/v3/projects/0a8684d9ff7f449295364dff438d34d5 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:43:28.756380 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f261681-fc32-4f4f-8e08-c467ff0d6e94 service neutron] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:28.766848 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f261681-fc32-4f4f-8e08-c467ff0d6e94 service neutron] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:28.823610 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9758cfc-9eb0-4b0f-bb26-4b8ed7847d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:28.824290 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9758cfc-9eb0-4b0f-bb26-4b8ed7847d90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:28.824579 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9758cfc-9eb0-4b0f-bb26-4b8ed7847d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:28.824855 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9758cfc-9eb0-4b0f-bb26-4b8ed7847d90 admin admin] PATH_INFO: `/v3/users/4294003b410b4b5da4b8d0b4aa839593` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:28.826021 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9758cfc-9eb0-4b0f-bb26-4b8ed7847d90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4294003b410b4b5da4b8d0b4aa839593)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:28.828663 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9758cfc-9eb0-4b0f-bb26-4b8ed7847d90 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:28.978313 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-e9758cfc-9eb0-4b0f-bb26-4b8ed7847d90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4294003b410b4b5da4b8d0b4aa839593'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:28.978908 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-e9758cfc-9eb0-4b0f-bb26-4b8ed7847d90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4294003b410b4b5da4b8d0b4aa839593'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:28.990872 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-e9758cfc-9eb0-4b0f-bb26-4b8ed7847d90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4294003b410b4b5da4b8d0b4aa839593'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:29.066646 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-e9758cfc-9eb0-4b0f-bb26-4b8ed7847d90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4294003b410b4b5da4b8d0b4aa839593'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:29.073884 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-e9758cfc-9eb0-4b0f-bb26-4b8ed7847d90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4294003b410b4b5da4b8d0b4aa839593'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:29.080458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1411/2802] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:43:28 2022] DELETE /identity/v3/users/4294003b410b4b5da4b8d0b4aa839593 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:43:29.493556 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c92a97-4589-4802-8804-13679c3b781d admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:29.495138 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-d4c92a97-4589-4802-8804-13679c3b781d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:29.515670 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c92a97-4589-4802-8804-13679c3b781d admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:29.529308 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:43:29.577646 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd0f1c9-7839-4907-b967-b61972cca9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:29.578501 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0f1c9-7839-4907-b967-b61972cca9b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:29.578883 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0f1c9-7839-4907-b967-b61972cca9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:29.579267 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0f1c9-7839-4907-b967-b61972cca9b3 admin admin] PATH_INFO: `/v3/projects/c7b2fe0310f0422c8922eefa3c88cf6f` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:29.580617 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd0f1c9-7839-4907-b967-b61972cca9b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7b2fe0310f0422c8922eefa3c88cf6f)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:29.582554 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd0f1c9-7839-4907-b967-b61972cca9b3 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:29.679975 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-6cd0f1c9-7839-4907-b967-b61972cca9b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7b2fe0310f0422c8922eefa3c88cf6f', 'request_id': 'req-6cd0f1c9-7839-4907-b967-b61972cca9b3'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:29.686973 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-6cd0f1c9-7839-4907-b967-b61972cca9b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7b2fe0310f0422c8922eefa3c88cf6f', 'request_id': 'req-6cd0f1c9-7839-4907-b967-b61972cca9b3'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:29.729593 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-6cd0f1c9-7839-4907-b967-b61972cca9b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7b2fe0310f0422c8922eefa3c88cf6f', 'request_id': 'req-6cd0f1c9-7839-4907-b967-b61972cca9b3'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:29.740678 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-6cd0f1c9-7839-4907-b967-b61972cca9b3 admin admin] The token cache is being invalidate because project c7b2fe0310f0422c8922eefa3c88cf6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:43:29.740914 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-6cd0f1c9-7839-4907-b967-b61972cca9b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:29.741564 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-6cd0f1c9-7839-4907-b967-b61972cca9b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:29.743060 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1392/2803] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:43:29 2022] DELETE /identity/v3/projects/c7b2fe0310f0422c8922eefa3c88cf6f => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:43:29.747948 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a00ee-3d84-4898-9792-95b1b89f9071 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:29.748264 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a00ee-3d84-4898-9792-95b1b89f9071 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:29.748544 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a00ee-3d84-4898-9792-95b1b89f9071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:29.768699 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-367a00ee-3d84-4898-9792-95b1b89f9071 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:43:29.770073 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-367a00ee-3d84-4898-9792-95b1b89f9071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:29.831663 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1412/2804] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:43:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:43:29.839512 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188047c5-aa09-4fb0-8db3-1c03400de3f2 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:29.839956 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188047c5-aa09-4fb0-8db3-1c03400de3f2 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:29.840340 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188047c5-aa09-4fb0-8db3-1c03400de3f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:29.861123 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-188047c5-aa09-4fb0-8db3-1c03400de3f2 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:43:29.862355 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-188047c5-aa09-4fb0-8db3-1c03400de3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:29.870704 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1393/2805] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:43:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:43:29.874996 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a00ee-3d84-4898-9792-95b1b89f9071 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:29.884736 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a00ee-3d84-4898-9792-95b1b89f9071 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:29.889320 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943930fe-d6b9-442f-b9d0-bb849dc85b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:29.889914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943930fe-d6b9-442f-b9d0-bb849dc85b96 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:29.890195 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943930fe-d6b9-442f-b9d0-bb849dc85b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:29.890470 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943930fe-d6b9-442f-b9d0-bb849dc85b96 admin admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:29.891157 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943930fe-d6b9-442f-b9d0-bb849dc85b96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:43:29.891461 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943930fe-d6b9-442f-b9d0-bb849dc85b96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:29.893078 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943930fe-d6b9-442f-b9d0-bb849dc85b96 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:29.899553 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:43:29.905099 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1413/2806] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:43:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:43:29.911039 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188047c5-aa09-4fb0-8db3-1c03400de3f2 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:29.920881 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188047c5-aa09-4fb0-8db3-1c03400de3f2 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:29.925590 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5030b6b4-df38-4bfe-b7ef-e753e5427b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:29.926193 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5030b6b4-df38-4bfe-b7ef-e753e5427b4b admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:29.926475 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5030b6b4-df38-4bfe-b7ef-e753e5427b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:29.926751 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5030b6b4-df38-4bfe-b7ef-e753e5427b4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:29.927407 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5030b6b4-df38-4bfe-b7ef-e753e5427b4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:29.929337 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5030b6b4-df38-4bfe-b7ef-e753e5427b4b admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:30.004906 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1394/2807] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:43:29 2022] POST /identity/v3/projects => generated 381 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:43:30.009021 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943930fe-d6b9-442f-b9d0-bb849dc85b96 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:30.019096 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943930fe-d6b9-442f-b9d0-bb849dc85b96 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:30.075021 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb86c67-2995-4ab7-8ec6-ad017bf18f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:30.075623 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb86c67-2995-4ab7-8ec6-ad017bf18f42 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:30.075905 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb86c67-2995-4ab7-8ec6-ad017bf18f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:30.076220 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb86c67-2995-4ab7-8ec6-ad017bf18f42 admin admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:30.076911 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb86c67-2995-4ab7-8ec6-ad017bf18f42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:30.078822 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb86c67-2995-4ab7-8ec6-ad017bf18f42 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:30.174404 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1414/2808] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:43:30 2022] POST /identity/v3/users => generated 291 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:43:30.178440 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5030b6b4-df38-4bfe-b7ef-e753e5427b4b admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:30.189018 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5030b6b4-df38-4bfe-b7ef-e753e5427b4b admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:30.193636 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8c6b7-caba-4b0e-bcb6-2cf5067eb338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:30.194240 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8c6b7-caba-4b0e-bcb6-2cf5067eb338 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:30.194558 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8c6b7-caba-4b0e-bcb6-2cf5067eb338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:30.194839 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8c6b7-caba-4b0e-bcb6-2cf5067eb338 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:30.195530 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af8c6b7-caba-4b0e-bcb6-2cf5067eb338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:30.197160 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af8c6b7-caba-4b0e-bcb6-2cf5067eb338 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:30.207857 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1395/2809] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:43:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:43:30.211974 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb86c67-2995-4ab7-8ec6-ad017bf18f42 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:30.222608 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb86c67-2995-4ab7-8ec6-ad017bf18f42 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:30.227377 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd9f4b0-b765-4aa0-8119-de30429235ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:30.227983 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9f4b0-b765-4aa0-8119-de30429235ca admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:30.228307 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9f4b0-b765-4aa0-8119-de30429235ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:30.228592 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9f4b0-b765-4aa0-8119-de30429235ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:30.229284 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd9f4b0-b765-4aa0-8119-de30429235ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:30.230919 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd9f4b0-b765-4aa0-8119-de30429235ca admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:30.241467 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1415/2810] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:43:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:43:30.245389 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8c6b7-caba-4b0e-bcb6-2cf5067eb338 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:30.255383 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8c6b7-caba-4b0e-bcb6-2cf5067eb338 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:30.259979 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c3425-f541-4b1f-a514-ca8afa1f163d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:30.260661 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c3425-f541-4b1f-a514-ca8afa1f163d admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:30.260943 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c3425-f541-4b1f-a514-ca8afa1f163d admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:30.261219 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c3425-f541-4b1f-a514-ca8afa1f163d admin admin] PATH_INFO: `/v3/projects/4243ceec64574e4e8a12a3daafd6dbf4/users/64db1402bd154e83b2d95b46245728c2/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:30.276125 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3c3425-f541-4b1f-a514-ca8afa1f163d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4243ceec64574e4e8a12a3daafd6dbf4, user_id=64db1402bd154e83b2d95b46245728c2, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:30.278735 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3c3425-f541-4b1f-a514-ca8afa1f163d admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:30.345896 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1396/2811] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:43:30 2022] PUT /identity/v3/projects/4243ceec64574e4e8a12a3daafd6dbf4/users/64db1402bd154e83b2d95b46245728c2/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:43:30.357145 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43286cdf-246c-429a-8c40-cc947abc5d3f None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:30.357438 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43286cdf-246c-429a-8c40-cc947abc5d3f None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:30.357746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43286cdf-246c-429a-8c40-cc947abc5d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:30.384629 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-43286cdf-246c-429a-8c40-cc947abc5d3f None None] MFA Rules not processed for user `64db1402bd154e83b2d95b46245728c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:43:30.385835 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-43286cdf-246c-429a-8c40-cc947abc5d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:30.448118 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1416/2812] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:43:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:43:30.453191 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c3425-f541-4b1f-a514-ca8afa1f163d admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:30.497409 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:43:30.513270 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c3425-f541-4b1f-a514-ca8afa1f163d admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:30.518471 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ba97f-457f-4123-a0a1-154d8652c630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:30.519074 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ba97f-457f-4123-a0a1-154d8652c630 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:30.519355 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ba97f-457f-4123-a0a1-154d8652c630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:30.519630 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ba97f-457f-4123-a0a1-154d8652c630 admin admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:30.520315 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5ba97f-457f-4123-a0a1-154d8652c630 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:30.522192 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5ba97f-457f-4123-a0a1-154d8652c630 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:30.621847 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1397/2813] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:43:30 2022] POST /identity/v3/projects => generated 379 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:43:30.626477 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43286cdf-246c-429a-8c40-cc947abc5d3f None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:30.637230 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43286cdf-246c-429a-8c40-cc947abc5d3f None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:30.700637 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db65a087-4640-4872-9be8-e7418f55c3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:30.701257 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65a087-4640-4872-9be8-e7418f55c3b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:30.702211 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65a087-4640-4872-9be8-e7418f55c3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:30.702495 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65a087-4640-4872-9be8-e7418f55c3b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:30.703158 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db65a087-4640-4872-9be8-e7418f55c3b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:30.705160 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db65a087-4640-4872-9be8-e7418f55c3b1 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:30.715873 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:43:30.787085 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1417/2814] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:43:30 2022] POST /identity/v3/users => generated 296 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:43:30.791223 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ba97f-457f-4123-a0a1-154d8652c630 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:30.801736 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ba97f-457f-4123-a0a1-154d8652c630 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:30.806442 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b975cc5-4a72-46d6-9633-895a76f6ae37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:30.807049 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b975cc5-4a72-46d6-9633-895a76f6ae37 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:30.807333 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b975cc5-4a72-46d6-9633-895a76f6ae37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:30.807615 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b975cc5-4a72-46d6-9633-895a76f6ae37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:30.808335 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b975cc5-4a72-46d6-9633-895a76f6ae37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:30.809922 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b975cc5-4a72-46d6-9633-895a76f6ae37 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:30.820439 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1398/2815] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:43:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:43:30.824375 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db65a087-4640-4872-9be8-e7418f55c3b1 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:30.835028 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db65a087-4640-4872-9be8-e7418f55c3b1 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:30.839721 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00cbb18-316a-4cea-8cff-fd7d52d9e601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:30.840433 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00cbb18-316a-4cea-8cff-fd7d52d9e601 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:30.840711 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00cbb18-316a-4cea-8cff-fd7d52d9e601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:30.840986 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00cbb18-316a-4cea-8cff-fd7d52d9e601 admin admin] PATH_INFO: `/v3/projects/51bf799f177749a3a81694b92dc7c132/users/cd15754f7a8b439bb58f55759dd91eff/roles/7c88c815db854954b516ab97cd471647` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:30.855751 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00cbb18-316a-4cea-8cff-fd7d52d9e601 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51bf799f177749a3a81694b92dc7c132, user_id=cd15754f7a8b439bb58f55759dd91eff, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:30.859015 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00cbb18-316a-4cea-8cff-fd7d52d9e601 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:30.935479 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1418/2816] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:43:30 2022] PUT /identity/v3/projects/51bf799f177749a3a81694b92dc7c132/users/cd15754f7a8b439bb58f55759dd91eff/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:43:30.939994 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b975cc5-4a72-46d6-9633-895a76f6ae37 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:30.950042 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b975cc5-4a72-46d6-9633-895a76f6ae37 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:31.006355 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1da048-2e0d-412b-85df-f671d8c933ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:31.007158 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1da048-2e0d-412b-85df-f671d8c933ab admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:31.007476 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1da048-2e0d-412b-85df-f671d8c933ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:31.007761 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1da048-2e0d-412b-85df-f671d8c933ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:31.008512 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1da048-2e0d-412b-85df-f671d8c933ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:31.010193 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1da048-2e0d-412b-85df-f671d8c933ab admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:31.021649 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1399/2817] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:43:30 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:43:31.025950 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00cbb18-316a-4cea-8cff-fd7d52d9e601 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:31.038040 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00cbb18-316a-4cea-8cff-fd7d52d9e601 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:31.042650 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8154e3-5723-4836-8de7-dd86e17b85d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:31.043309 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8154e3-5723-4836-8de7-dd86e17b85d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:31.043592 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8154e3-5723-4836-8de7-dd86e17b85d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:31.043871 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8154e3-5723-4836-8de7-dd86e17b85d4 admin admin] PATH_INFO: `/v3/projects/51bf799f177749a3a81694b92dc7c132/users/cd15754f7a8b439bb58f55759dd91eff/roles/7c88c815db854954b516ab97cd471647` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:31.046389 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8154e3-5723-4836-8de7-dd86e17b85d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51bf799f177749a3a81694b92dc7c132, user_id=cd15754f7a8b439bb58f55759dd91eff, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:31.049037 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8154e3-5723-4836-8de7-dd86e17b85d4 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:31.059279 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1419/2818] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:43:31 2022] PUT /identity/v3/projects/51bf799f177749a3a81694b92dc7c132/users/cd15754f7a8b439bb58f55759dd91eff/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:43:31.070211 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a517f1-ac92-48d0-a667-6bc91fed6488 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:31.075317 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a517f1-ac92-48d0-a667-6bc91fed6488 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:31.075317 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a517f1-ac92-48d0-a667-6bc91fed6488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:31.098118 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-49a517f1-ac92-48d0-a667-6bc91fed6488 None None] MFA Rules not processed for user `cd15754f7a8b439bb58f55759dd91eff`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:43:31.099327 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-49a517f1-ac92-48d0-a667-6bc91fed6488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:31.166041 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1400/2819] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:43:31 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:43:31.179161 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8154e3-5723-4836-8de7-dd86e17b85d4 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:31.180926 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-ec8154e3-5723-4836-8de7-dd86e17b85d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:31.238766 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8154e3-5723-4836-8de7-dd86e17b85d4 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:31.242421 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaee0fc-03c0-4cd9-813a-da77264594a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:31.243031 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaee0fc-03c0-4cd9-813a-da77264594a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:31.243319 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaee0fc-03c0-4cd9-813a-da77264594a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:31.243600 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaee0fc-03c0-4cd9-813a-da77264594a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:31.252539 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaee0fc-03c0-4cd9-813a-da77264594a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:31.254008 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaee0fc-03c0-4cd9-813a-da77264594a5 service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:31.263354 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1420/2820] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:43:31 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:43:31.885532 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a517f1-ac92-48d0-a667-6bc91fed6488 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:31.929427 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:43:31.949105 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a517f1-ac92-48d0-a667-6bc91fed6488 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:31.954369 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d6116-04d0-40c3-8dd5-b603d7e8d962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:31.955083 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d6116-04d0-40c3-8dd5-b603d7e8d962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:31.955395 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d6116-04d0-40c3-8dd5-b603d7e8d962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:31.955671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d6116-04d0-40c3-8dd5-b603d7e8d962 admin admin] PATH_INFO: `/v3/users/64db1402bd154e83b2d95b46245728c2` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:31.956926 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03d6116-04d0-40c3-8dd5-b603d7e8d962 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64db1402bd154e83b2d95b46245728c2)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:31.958879 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03d6116-04d0-40c3-8dd5-b603d7e8d962 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:32.148955 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-c03d6116-04d0-40c3-8dd5-b603d7e8d962 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64db1402bd154e83b2d95b46245728c2'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:32.155976 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-c03d6116-04d0-40c3-8dd5-b603d7e8d962 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64db1402bd154e83b2d95b46245728c2'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:32.159646 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-c03d6116-04d0-40c3-8dd5-b603d7e8d962 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64db1402bd154e83b2d95b46245728c2'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:32.173022 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-c03d6116-04d0-40c3-8dd5-b603d7e8d962 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64db1402bd154e83b2d95b46245728c2'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:32.173828 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-c03d6116-04d0-40c3-8dd5-b603d7e8d962 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64db1402bd154e83b2d95b46245728c2'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:32.249176 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1401/2821] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:43:31 2022] DELETE /identity/v3/users/64db1402bd154e83b2d95b46245728c2 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:43:32.254387 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a57e2b-4a5a-4dd2-a223-840bd78b60ad None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:32.254873 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a57e2b-4a5a-4dd2-a223-840bd78b60ad None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:32.254873 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a57e2b-4a5a-4dd2-a223-840bd78b60ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:32.273992 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-82a57e2b-4a5a-4dd2-a223-840bd78b60ad None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:43:32.275208 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-82a57e2b-4a5a-4dd2-a223-840bd78b60ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:32.336635 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1421/2822] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:43:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:43:32.346926 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d6116-04d0-40c3-8dd5-b603d7e8d962 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:32.348454 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-c03d6116-04d0-40c3-8dd5-b603d7e8d962 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:32.411065 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d6116-04d0-40c3-8dd5-b603d7e8d962 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:32.411065 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfde38c-eed5-4371-8e6b-ea3742f2979a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:32.411065 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfde38c-eed5-4371-8e6b-ea3742f2979a service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:32.411065 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfde38c-eed5-4371-8e6b-ea3742f2979a service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:32.411538 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfde38c-eed5-4371-8e6b-ea3742f2979a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:32.419877 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfde38c-eed5-4371-8e6b-ea3742f2979a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:32.421868 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfde38c-eed5-4371-8e6b-ea3742f2979a service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:32.431895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1402/2823] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:43:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:43:32.822357 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a57e2b-4a5a-4dd2-a223-840bd78b60ad None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:32.823898 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-82a57e2b-4a5a-4dd2-a223-840bd78b60ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:32.836878 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a57e2b-4a5a-4dd2-a223-840bd78b60ad None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:32.841778 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa5a16-dbbb-42f0-ba86-8f591647c580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:32.842489 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa5a16-dbbb-42f0-ba86-8f591647c580 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:32.842782 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa5a16-dbbb-42f0-ba86-8f591647c580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:32.843681 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa5a16-dbbb-42f0-ba86-8f591647c580 admin admin] PATH_INFO: `/v3/projects/4243ceec64574e4e8a12a3daafd6dbf4` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:32.845081 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa5a16-dbbb-42f0-ba86-8f591647c580 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4243ceec64574e4e8a12a3daafd6dbf4)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:32.846880 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa5a16-dbbb-42f0-ba86-8f591647c580 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:32.952255 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-eeaa5a16-dbbb-42f0-ba86-8f591647c580 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4243ceec64574e4e8a12a3daafd6dbf4', 'request_id': 'req-eeaa5a16-dbbb-42f0-ba86-8f591647c580'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:32.958527 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-eeaa5a16-dbbb-42f0-ba86-8f591647c580 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4243ceec64574e4e8a12a3daafd6dbf4', 'request_id': 'req-eeaa5a16-dbbb-42f0-ba86-8f591647c580'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:32.960614 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:43:32.964469 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-eeaa5a16-dbbb-42f0-ba86-8f591647c580 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4243ceec64574e4e8a12a3daafd6dbf4', 'request_id': 'req-eeaa5a16-dbbb-42f0-ba86-8f591647c580'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:33.038435 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-eeaa5a16-dbbb-42f0-ba86-8f591647c580 admin admin] The token cache is being invalidate because project 4243ceec64574e4e8a12a3daafd6dbf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:43:33.038724 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-eeaa5a16-dbbb-42f0-ba86-8f591647c580 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:33.039343 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-eeaa5a16-dbbb-42f0-ba86-8f591647c580 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:33.040882 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1422/2824] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:43:32 2022] DELETE /identity/v3/projects/4243ceec64574e4e8a12a3daafd6dbf4 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:43:33.047422 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfde38c-eed5-4371-8e6b-ea3742f2979a service neutron] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:33.057480 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfde38c-eed5-4371-8e6b-ea3742f2979a service neutron] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:33.114272 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea1d505-f82e-42ad-a428-2b10e50bc360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:33.114902 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea1d505-f82e-42ad-a428-2b10e50bc360 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:33.115183 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea1d505-f82e-42ad-a428-2b10e50bc360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:33.115459 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea1d505-f82e-42ad-a428-2b10e50bc360 admin admin] PATH_INFO: `/v3/users/cd15754f7a8b439bb58f55759dd91eff` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:33.116638 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea1d505-f82e-42ad-a428-2b10e50bc360 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd15754f7a8b439bb58f55759dd91eff)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:33.118534 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea1d505-f82e-42ad-a428-2b10e50bc360 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:33.120942 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:43:33.300931 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-8ea1d505-f82e-42ad-a428-2b10e50bc360 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd15754f7a8b439bb58f55759dd91eff'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:33.307719 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-8ea1d505-f82e-42ad-a428-2b10e50bc360 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd15754f7a8b439bb58f55759dd91eff'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:33.308559 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-8ea1d505-f82e-42ad-a428-2b10e50bc360 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd15754f7a8b439bb58f55759dd91eff'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:33.320010 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-8ea1d505-f82e-42ad-a428-2b10e50bc360 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd15754f7a8b439bb58f55759dd91eff'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:33.320672 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-8ea1d505-f82e-42ad-a428-2b10e50bc360 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd15754f7a8b439bb58f55759dd91eff'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:33.396520 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1403/2825] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:43:33 2022] DELETE /identity/v3/users/cd15754f7a8b439bb58f55759dd91eff => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:43:33.766253 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa5a16-dbbb-42f0-ba86-8f591647c580 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:33.768252 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-eeaa5a16-dbbb-42f0-ba86-8f591647c580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:33.781177 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa5a16-dbbb-42f0-ba86-8f591647c580 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:33.838548 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f605ba-0a67-43ab-ad0c-41c91e874839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:33.839132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f605ba-0a67-43ab-ad0c-41c91e874839 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:33.839443 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f605ba-0a67-43ab-ad0c-41c91e874839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:33.839751 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f605ba-0a67-43ab-ad0c-41c91e874839 admin admin] PATH_INFO: `/v3/projects/51bf799f177749a3a81694b92dc7c132` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:33.841043 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f605ba-0a67-43ab-ad0c-41c91e874839 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51bf799f177749a3a81694b92dc7c132)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:33.842960 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f605ba-0a67-43ab-ad0c-41c91e874839 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:33.952798 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-06f605ba-0a67-43ab-ad0c-41c91e874839 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51bf799f177749a3a81694b92dc7c132', 'request_id': 'req-06f605ba-0a67-43ab-ad0c-41c91e874839'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:33.958932 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-06f605ba-0a67-43ab-ad0c-41c91e874839 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51bf799f177749a3a81694b92dc7c132', 'request_id': 'req-06f605ba-0a67-43ab-ad0c-41c91e874839'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:33.964709 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-06f605ba-0a67-43ab-ad0c-41c91e874839 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51bf799f177749a3a81694b92dc7c132', 'request_id': 'req-06f605ba-0a67-43ab-ad0c-41c91e874839'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:34.043554 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-06f605ba-0a67-43ab-ad0c-41c91e874839 admin admin] The token cache is being invalidate because project 51bf799f177749a3a81694b92dc7c132 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:43:34.043847 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-06f605ba-0a67-43ab-ad0c-41c91e874839 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:34.044494 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-06f605ba-0a67-43ab-ad0c-41c91e874839 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:34.046657 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1423/2826] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:43:33 2022] DELETE /identity/v3/projects/51bf799f177749a3a81694b92dc7c132 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:43:34.051383 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf57d6-d06b-41c5-bc46-050d5a4ea173 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:34.051879 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf57d6-d06b-41c5-bc46-050d5a4ea173 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:34.051997 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf57d6-d06b-41c5-bc46-050d5a4ea173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:34.072415 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-fbdf57d6-d06b-41c5-bc46-050d5a4ea173 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:43:34.073628 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-fbdf57d6-d06b-41c5-bc46-050d5a4ea173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:34.133139 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1404/2827] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:43:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:43:34.142055 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57a9b3-3ad8-46b1-b18e-8aa5b948b2a9 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:34.142344 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57a9b3-3ad8-46b1-b18e-8aa5b948b2a9 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:34.142623 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57a9b3-3ad8-46b1-b18e-8aa5b948b2a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:34.162355 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-5e57a9b3-3ad8-46b1-b18e-8aa5b948b2a9 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:43:34.163554 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-5e57a9b3-3ad8-46b1-b18e-8aa5b948b2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:34.172347 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1424/2828] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:43:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:43:34.176549 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf57d6-d06b-41c5-bc46-050d5a4ea173 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:34.186500 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf57d6-d06b-41c5-bc46-050d5a4ea173 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:34.191838 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504e0cbd-fc0c-4f61-bc69-23428a63481a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:34.192487 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e0cbd-fc0c-4f61-bc69-23428a63481a admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:34.192773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e0cbd-fc0c-4f61-bc69-23428a63481a admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:34.193052 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e0cbd-fc0c-4f61-bc69-23428a63481a admin admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:34.193776 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504e0cbd-fc0c-4f61-bc69-23428a63481a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:43:34.194090 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504e0cbd-fc0c-4f61-bc69-23428a63481a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:34.195684 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504e0cbd-fc0c-4f61-bc69-23428a63481a admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:34.207247 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1405/2829] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:43:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:43:34.211577 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e57a9b3-3ad8-46b1-b18e-8aa5b948b2a9 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:34.221233 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e57a9b3-3ad8-46b1-b18e-8aa5b948b2a9 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:34.226106 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d42f5c-4941-48a7-8f0d-0bc5e211ce7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:34.226758 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d42f5c-4941-48a7-8f0d-0bc5e211ce7f admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:34.227060 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d42f5c-4941-48a7-8f0d-0bc5e211ce7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:34.227338 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d42f5c-4941-48a7-8f0d-0bc5e211ce7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:34.228111 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d42f5c-4941-48a7-8f0d-0bc5e211ce7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:34.230012 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d42f5c-4941-48a7-8f0d-0bc5e211ce7f admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:34.341211 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1425/2830] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:43:34 2022] POST /identity/v3/projects => generated 377 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:43:34.342441 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504e0cbd-fc0c-4f61-bc69-23428a63481a admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:34.352489 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504e0cbd-fc0c-4f61-bc69-23428a63481a admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:34.411062 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31180d2-cc1f-4fef-90d9-95f2f43afa81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:34.411683 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31180d2-cc1f-4fef-90d9-95f2f43afa81 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:34.411969 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31180d2-cc1f-4fef-90d9-95f2f43afa81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:34.412320 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31180d2-cc1f-4fef-90d9-95f2f43afa81 admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:34.412988 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31180d2-cc1f-4fef-90d9-95f2f43afa81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:34.414893 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31180d2-cc1f-4fef-90d9-95f2f43afa81 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:34.484925 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:43:34.494249 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1406/2831] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:43:34 2022] POST /identity/v3/users => generated 289 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:43:34.495588 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d42f5c-4941-48a7-8f0d-0bc5e211ce7f admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:34.506785 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d42f5c-4941-48a7-8f0d-0bc5e211ce7f admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:34.512684 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d216388a-b891-49c1-8e41-b1a8c10de91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:34.513306 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d216388a-b891-49c1-8e41-b1a8c10de91b admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:34.513591 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d216388a-b891-49c1-8e41-b1a8c10de91b admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:34.513870 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d216388a-b891-49c1-8e41-b1a8c10de91b admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:34.514568 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d216388a-b891-49c1-8e41-b1a8c10de91b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:34.516828 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d216388a-b891-49c1-8e41-b1a8c10de91b admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:34.526941 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1426/2832] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:43:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:43:34.531025 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31180d2-cc1f-4fef-90d9-95f2f43afa81 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:34.541801 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31180d2-cc1f-4fef-90d9-95f2f43afa81 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:34.547909 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179723ce-3362-4bfe-bf20-2a225f83a0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:34.548579 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179723ce-3362-4bfe-bf20-2a225f83a0dd admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:34.548888 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179723ce-3362-4bfe-bf20-2a225f83a0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:34.549152 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179723ce-3362-4bfe-bf20-2a225f83a0dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:34.549881 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179723ce-3362-4bfe-bf20-2a225f83a0dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:34.551510 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179723ce-3362-4bfe-bf20-2a225f83a0dd admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:34.563029 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1407/2833] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:43:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 15:43:34.567662 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d216388a-b891-49c1-8e41-b1a8c10de91b admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:34.578581 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d216388a-b891-49c1-8e41-b1a8c10de91b admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:34.583691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe13544-e8b3-4d1e-99ff-7370d1fcd269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:34.584383 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe13544-e8b3-4d1e-99ff-7370d1fcd269 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:34.584666 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe13544-e8b3-4d1e-99ff-7370d1fcd269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:34.584945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe13544-e8b3-4d1e-99ff-7370d1fcd269 admin admin] PATH_INFO: `/v3/projects/804c691d29234f53bee590edd642ad97/users/f306f7db6bd44443b457eb70d257e004/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:34.601563 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe13544-e8b3-4d1e-99ff-7370d1fcd269 admin admin] RBAC: Authorizing `identity:create_grant(project_id=804c691d29234f53bee590edd642ad97, user_id=f306f7db6bd44443b457eb70d257e004, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:34.604271 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe13544-e8b3-4d1e-99ff-7370d1fcd269 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:34.680203 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1427/2834] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:43:34 2022] PUT /identity/v3/projects/804c691d29234f53bee590edd642ad97/users/f306f7db6bd44443b457eb70d257e004/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:43:34.691902 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc99fe-8988-4dec-81f8-9a29982e80ba None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:34.692230 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc99fe-8988-4dec-81f8-9a29982e80ba None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:34.692505 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc99fe-8988-4dec-81f8-9a29982e80ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:34.721711 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-c8cc99fe-8988-4dec-81f8-9a29982e80ba None None] MFA Rules not processed for user `f306f7db6bd44443b457eb70d257e004`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:43:34.722959 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-c8cc99fe-8988-4dec-81f8-9a29982e80ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:34.792502 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1408/2835] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:43:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:43:34.804440 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe13544-e8b3-4d1e-99ff-7370d1fcd269 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:34.806224 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-abe13544-e8b3-4d1e-99ff-7370d1fcd269 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:34.820923 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:43:34.866424 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe13544-e8b3-4d1e-99ff-7370d1fcd269 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:34.870259 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e5176-81c1-436a-ad74-8b3df13ffd41 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c386fa0483cf42efaddbc2395f6d1fce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:34.870893 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e5176-81c1-436a-ad74-8b3df13ffd41 service glance] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:34.871179 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e5176-81c1-436a-ad74-8b3df13ffd41 service glance] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:34.871458 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e5176-81c1-436a-ad74-8b3df13ffd41 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:34.881263 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662e5176-81c1-436a-ad74-8b3df13ffd41 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:34.882775 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662e5176-81c1-436a-ad74-8b3df13ffd41 service glance] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:34.892939 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1428/2836] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:43:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:43:43.363280 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8329c70-5cc8-419c-be99-3b4d13089f0d None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:43.363779 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8329c70-5cc8-419c-be99-3b4d13089f0d None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:43.363875 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8329c70-5cc8-419c-be99-3b4d13089f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:43.386447 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-c8329c70-5cc8-419c-be99-3b4d13089f0d None None] MFA Rules not processed for user `1654fb6a81e14d6b99ab495bb6d029d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:43:43.387715 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-c8329c70-5cc8-419c-be99-3b4d13089f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:43.447040 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1409/2837] 10.222.0.40 () {58 vars in 967 bytes} [Fri Aug 5 15:43:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:43:43.487539 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e5176-81c1-436a-ad74-8b3df13ffd41 service glance] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:43.489020 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-662e5176-81c1-436a-ad74-8b3df13ffd41 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:43.559347 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e5176-81c1-436a-ad74-8b3df13ffd41 service glance] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:43.564945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199b383-5345-4098-90c4-e78128bac31d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85154ce7446245738edcc8702dbf6861', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:43.565565 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199b383-5345-4098-90c4-e78128bac31d service swift] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:43.565853 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199b383-5345-4098-90c4-e78128bac31d service swift] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:43.566132 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199b383-5345-4098-90c4-e78128bac31d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:43.574129 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c199b383-5345-4098-90c4-e78128bac31d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:43.575707 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c199b383-5345-4098-90c4-e78128bac31d service swift] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:43.584735 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1429/2838] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 5 15:43:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 15:43:47.599101 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8329c70-5cc8-419c-be99-3b4d13089f0d None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:47.600908 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-c8329c70-5cc8-419c-be99-3b4d13089f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:47.674507 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8329c70-5cc8-419c-be99-3b4d13089f0d None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:47.679817 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd5262-9b23-4040-b2dc-ec7be7025508 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '107a7c18a2c6440b86e1b7c3e39cc724', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:47.680492 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd5262-9b23-4040-b2dc-ec7be7025508 service placement] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:47.680820 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd5262-9b23-4040-b2dc-ec7be7025508 service placement] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:47.681074 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd5262-9b23-4040-b2dc-ec7be7025508 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:47.682974 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-02fd5262-9b23-4040-b2dc-ec7be7025508 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:47.696520 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd5262-9b23-4040-b2dc-ec7be7025508 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:47.698329 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd5262-9b23-4040-b2dc-ec7be7025508 service placement] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:47.709252 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1410/2839] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:43:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:43:51.639236 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199b383-5345-4098-90c4-e78128bac31d service swift] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:51.641325 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-c199b383-5345-4098-90c4-e78128bac31d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:51.711691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199b383-5345-4098-90c4-e78128bac31d service swift] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:51.716513 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72de4e7-2a96-4f9c-be8f-7050ce43acb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:51.720499 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72de4e7-2a96-4f9c-be8f-7050ce43acb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:51.720499 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72de4e7-2a96-4f9c-be8f-7050ce43acb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:51.720499 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72de4e7-2a96-4f9c-be8f-7050ce43acb4 admin admin] PATH_INFO: `/v3/users/8ffd179bd4a445eaafb53d7accede863` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:51.720499 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72de4e7-2a96-4f9c-be8f-7050ce43acb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ffd179bd4a445eaafb53d7accede863)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:51.721164 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72de4e7-2a96-4f9c-be8f-7050ce43acb4 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:51.904891 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:43:51.906545 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-a72de4e7-2a96-4f9c-be8f-7050ce43acb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ffd179bd4a445eaafb53d7accede863'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:51.907202 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-a72de4e7-2a96-4f9c-be8f-7050ce43acb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ffd179bd4a445eaafb53d7accede863'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:51.919230 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-a72de4e7-2a96-4f9c-be8f-7050ce43acb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ffd179bd4a445eaafb53d7accede863'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:51.997935 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-a72de4e7-2a96-4f9c-be8f-7050ce43acb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ffd179bd4a445eaafb53d7accede863'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:52.005768 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-a72de4e7-2a96-4f9c-be8f-7050ce43acb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ffd179bd4a445eaafb53d7accede863'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:52.011763 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1430/2840] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:43:51 2022] DELETE /identity/v3/users/8ffd179bd4a445eaafb53d7accede863 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:43:52.279802 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd5262-9b23-4040-b2dc-ec7be7025508 service placement] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:52.292134 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd5262-9b23-4040-b2dc-ec7be7025508 service placement] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:52.300850 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=100 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:43:52.347583 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72de4e7-2a96-4f9c-be8f-7050ce43acb4 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:52.348999 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-a72de4e7-2a96-4f9c-be8f-7050ce43acb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:52.350175 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5600476a-b72c-417c-b3ae-42c0bd089712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:52.350827 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600476a-b72c-417c-b3ae-42c0bd089712 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:52.351119 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600476a-b72c-417c-b3ae-42c0bd089712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:52.351401 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600476a-b72c-417c-b3ae-42c0bd089712 admin admin] PATH_INFO: `/v3/projects/6e62e9372a7a47528e7973537bebbfb6` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:52.352632 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5600476a-b72c-417c-b3ae-42c0bd089712 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e62e9372a7a47528e7973537bebbfb6)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:52.354525 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5600476a-b72c-417c-b3ae-42c0bd089712 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:52.363415 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72de4e7-2a96-4f9c-be8f-7050ce43acb4 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:52.368233 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973be01a-c9c6-4c63-8451-24e3ff16c4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:52.368883 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973be01a-c9c6-4c63-8451-24e3ff16c4dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:52.369288 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973be01a-c9c6-4c63-8451-24e3ff16c4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:52.369577 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973be01a-c9c6-4c63-8451-24e3ff16c4dc admin admin] PATH_INFO: `/v3/users/f306f7db6bd44443b457eb70d257e004` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:52.370746 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973be01a-c9c6-4c63-8451-24e3ff16c4dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f306f7db6bd44443b457eb70d257e004)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:52.372836 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973be01a-c9c6-4c63-8451-24e3ff16c4dc admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:52.455195 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-5600476a-b72c-417c-b3ae-42c0bd089712 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e62e9372a7a47528e7973537bebbfb6', 'request_id': 'req-5600476a-b72c-417c-b3ae-42c0bd089712'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:52.461455 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-5600476a-b72c-417c-b3ae-42c0bd089712 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e62e9372a7a47528e7973537bebbfb6', 'request_id': 'req-5600476a-b72c-417c-b3ae-42c0bd089712'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:52.541029 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-5600476a-b72c-417c-b3ae-42c0bd089712 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e62e9372a7a47528e7973537bebbfb6', 'request_id': 'req-5600476a-b72c-417c-b3ae-42c0bd089712'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:52.552671 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-5600476a-b72c-417c-b3ae-42c0bd089712 admin admin] The token cache is being invalidate because project 6e62e9372a7a47528e7973537bebbfb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:43:52.552959 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-5600476a-b72c-417c-b3ae-42c0bd089712 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:52.553586 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-5600476a-b72c-417c-b3ae-42c0bd089712 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:52.555178 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1411/2841] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:43:52 2022] DELETE /identity/v3/projects/6e62e9372a7a47528e7973537bebbfb6 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:43:52.558942 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5600476a-b72c-417c-b3ae-42c0bd089712 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:52.560263 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-5600476a-b72c-417c-b3ae-42c0bd089712 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:52.567349 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-973be01a-c9c6-4c63-8451-24e3ff16c4dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f306f7db6bd44443b457eb70d257e004'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:52.568001 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-973be01a-c9c6-4c63-8451-24e3ff16c4dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f306f7db6bd44443b457eb70d257e004'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:52.581557 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-973be01a-c9c6-4c63-8451-24e3ff16c4dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f306f7db6bd44443b457eb70d257e004'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:52.625094 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5600476a-b72c-417c-b3ae-42c0bd089712 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:52.629892 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc22a54-c039-49aa-a564-f411bc200a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:52.630530 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc22a54-c039-49aa-a564-f411bc200a5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:52.630859 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc22a54-c039-49aa-a564-f411bc200a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:52.631136 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc22a54-c039-49aa-a564-f411bc200a5e admin admin] PATH_INFO: `/v3/users/acf6022c12a14426a39fdd7b9e9cebb1` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:52.632333 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc22a54-c039-49aa-a564-f411bc200a5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=acf6022c12a14426a39fdd7b9e9cebb1)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:52.634252 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc22a54-c039-49aa-a564-f411bc200a5e admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:52.643925 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-973be01a-c9c6-4c63-8451-24e3ff16c4dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f306f7db6bd44443b457eb70d257e004'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:52.651642 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-973be01a-c9c6-4c63-8451-24e3ff16c4dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f306f7db6bd44443b457eb70d257e004'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:52.658904 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1431/2842] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:43:52 2022] DELETE /identity/v3/users/f306f7db6bd44443b457eb70d257e004 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:43:52.663263 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3d856-7bed-448c-b26d-d6daa96491d1 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:52.663556 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3d856-7bed-448c-b26d-d6daa96491d1 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:52.663841 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3d856-7bed-448c-b26d-d6daa96491d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:52.683574 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-89c3d856-7bed-448c-b26d-d6daa96491d1 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:43:52.684814 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-89c3d856-7bed-448c-b26d-d6daa96491d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:52.742709 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1432/2843] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:43:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:43:52.752375 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c3d856-7bed-448c-b26d-d6daa96491d1 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:52.753692 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-89c3d856-7bed-448c-b26d-d6daa96491d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:52.794555 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:43:52.815097 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c3d856-7bed-448c-b26d-d6daa96491d1 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:52.815775 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:43:52.818343 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-4dc22a54-c039-49aa-a564-f411bc200a5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acf6022c12a14426a39fdd7b9e9cebb1'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:52.818965 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163bc429-974f-460a-96e2-d5edb09596e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:52.819588 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163bc429-974f-460a-96e2-d5edb09596e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:52.819971 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163bc429-974f-460a-96e2-d5edb09596e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:52.820278 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163bc429-974f-460a-96e2-d5edb09596e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:52.830171 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163bc429-974f-460a-96e2-d5edb09596e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:52.831641 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163bc429-974f-460a-96e2-d5edb09596e6 service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:52.835177 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-4dc22a54-c039-49aa-a564-f411bc200a5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acf6022c12a14426a39fdd7b9e9cebb1'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:52.835177 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-4dc22a54-c039-49aa-a564-f411bc200a5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acf6022c12a14426a39fdd7b9e9cebb1'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:52.841434 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1433/2844] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:43:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 15:43:52.848047 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-4dc22a54-c039-49aa-a564-f411bc200a5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acf6022c12a14426a39fdd7b9e9cebb1'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:52.848772 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-4dc22a54-c039-49aa-a564-f411bc200a5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acf6022c12a14426a39fdd7b9e9cebb1'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:52.962826 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1412/2845] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:43:52 2022] DELETE /identity/v3/users/acf6022c12a14426a39fdd7b9e9cebb1 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:43:53.251804 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163bc429-974f-460a-96e2-d5edb09596e6 service neutron] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:53.253273 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-163bc429-974f-460a-96e2-d5edb09596e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:53.266696 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163bc429-974f-460a-96e2-d5edb09596e6 service neutron] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:53.325626 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41e0f4-3aef-4351-9dc8-5a772e8ec7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:53.327507 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41e0f4-3aef-4351-9dc8-5a772e8ec7fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:53.327797 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41e0f4-3aef-4351-9dc8-5a772e8ec7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:53.328093 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41e0f4-3aef-4351-9dc8-5a772e8ec7fa admin admin] PATH_INFO: `/v3/projects/9cb1f083fca24a3a8bfaa6b085b02856` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:53.329293 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c41e0f4-3aef-4351-9dc8-5a772e8ec7fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cb1f083fca24a3a8bfaa6b085b02856)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:53.331201 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c41e0f4-3aef-4351-9dc8-5a772e8ec7fa admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:53.365521 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc22a54-c039-49aa-a564-f411bc200a5e admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:53.366228 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-4dc22a54-c039-49aa-a564-f411bc200a5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:53.380406 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc22a54-c039-49aa-a564-f411bc200a5e admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:53.385756 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ac4ad-06cf-4d9b-88e4-9a805920fe27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:53.386572 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ac4ad-06cf-4d9b-88e4-9a805920fe27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:53.386896 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ac4ad-06cf-4d9b-88e4-9a805920fe27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:53.387121 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ac4ad-06cf-4d9b-88e4-9a805920fe27 admin admin] PATH_INFO: `/v3/projects/804c691d29234f53bee590edd642ad97` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:53.388311 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83ac4ad-06cf-4d9b-88e4-9a805920fe27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=804c691d29234f53bee590edd642ad97)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:53.390392 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83ac4ad-06cf-4d9b-88e4-9a805920fe27 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:53.461437 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-6c41e0f4-3aef-4351-9dc8-5a772e8ec7fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cb1f083fca24a3a8bfaa6b085b02856', 'request_id': 'req-6c41e0f4-3aef-4351-9dc8-5a772e8ec7fa'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:53.467674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-6c41e0f4-3aef-4351-9dc8-5a772e8ec7fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cb1f083fca24a3a8bfaa6b085b02856', 'request_id': 'req-6c41e0f4-3aef-4351-9dc8-5a772e8ec7fa'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:53.474389 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-6c41e0f4-3aef-4351-9dc8-5a772e8ec7fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cb1f083fca24a3a8bfaa6b085b02856', 'request_id': 'req-6c41e0f4-3aef-4351-9dc8-5a772e8ec7fa'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:53.485346 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-a83ac4ad-06cf-4d9b-88e4-9a805920fe27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '804c691d29234f53bee590edd642ad97', 'request_id': 'req-a83ac4ad-06cf-4d9b-88e4-9a805920fe27'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:53.491616 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-a83ac4ad-06cf-4d9b-88e4-9a805920fe27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '804c691d29234f53bee590edd642ad97', 'request_id': 'req-a83ac4ad-06cf-4d9b-88e4-9a805920fe27'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:53.552548 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-6c41e0f4-3aef-4351-9dc8-5a772e8ec7fa admin admin] The token cache is being invalidate because project 9cb1f083fca24a3a8bfaa6b085b02856 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:43:53.552772 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-6c41e0f4-3aef-4351-9dc8-5a772e8ec7fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:53.553399 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-6c41e0f4-3aef-4351-9dc8-5a772e8ec7fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:53.554952 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1434/2846] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:43:53 2022] DELETE /identity/v3/projects/9cb1f083fca24a3a8bfaa6b085b02856 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:43:53.592919 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-a83ac4ad-06cf-4d9b-88e4-9a805920fe27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '804c691d29234f53bee590edd642ad97', 'request_id': 'req-a83ac4ad-06cf-4d9b-88e4-9a805920fe27'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:53.605194 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-a83ac4ad-06cf-4d9b-88e4-9a805920fe27 admin admin] The token cache is being invalidate because project 804c691d29234f53bee590edd642ad97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80541) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:43:53.605493 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-a83ac4ad-06cf-4d9b-88e4-9a805920fe27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:53.606204 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-a83ac4ad-06cf-4d9b-88e4-9a805920fe27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:43:53.607757 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1413/2847] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:43:53 2022] DELETE /identity/v3/projects/804c691d29234f53bee590edd642ad97 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:43:53.613315 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db771ba-8506-43eb-8757-73251a9ee8eb None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:53.613607 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db771ba-8506-43eb-8757-73251a9ee8eb None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:53.613895 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db771ba-8506-43eb-8757-73251a9ee8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:53.635651 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-0db771ba-8506-43eb-8757-73251a9ee8eb None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:43:53.637060 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-0db771ba-8506-43eb-8757-73251a9ee8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:53.701989 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1435/2848] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:43:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:43:53.710171 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e97af3-b8a8-4ce1-9b87-d8ab59eb01da None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:53.710486 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e97af3-b8a8-4ce1-9b87-d8ab59eb01da None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:53.710742 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e97af3-b8a8-4ce1-9b87-d8ab59eb01da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:53.732516 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-30e97af3-b8a8-4ce1-9b87-d8ab59eb01da None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:43:53.734455 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-30e97af3-b8a8-4ce1-9b87-d8ab59eb01da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:53.742906 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1414/2849] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:43:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:43:53.747223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db771ba-8506-43eb-8757-73251a9ee8eb None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:53.758063 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db771ba-8506-43eb-8757-73251a9ee8eb None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:53.763026 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d124d0b-fc22-46c0-9628-eeb20f07cb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:53.763665 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d124d0b-fc22-46c0-9628-eeb20f07cb0f admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:53.763955 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d124d0b-fc22-46c0-9628-eeb20f07cb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:53.764286 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d124d0b-fc22-46c0-9628-eeb20f07cb0f admin admin] PATH_INFO: `/v3/domains` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:53.765008 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d124d0b-fc22-46c0-9628-eeb20f07cb0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:43:53.765324 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d124d0b-fc22-46c0-9628-eeb20f07cb0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:53.767001 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d124d0b-fc22-46c0-9628-eeb20f07cb0f admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:53.770186 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:43:53.783917 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1436/2850] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:43:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:43:53.784992 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e97af3-b8a8-4ce1-9b87-d8ab59eb01da None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:53.794940 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e97af3-b8a8-4ce1-9b87-d8ab59eb01da None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:53.800714 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38df548-c885-4563-a5b9-0f0e69dc5c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:53.801352 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38df548-c885-4563-a5b9-0f0e69dc5c9f admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:53.801636 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38df548-c885-4563-a5b9-0f0e69dc5c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:53.802060 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38df548-c885-4563-a5b9-0f0e69dc5c9f admin admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:53.802809 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38df548-c885-4563-a5b9-0f0e69dc5c9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:53.804756 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38df548-c885-4563-a5b9-0f0e69dc5c9f admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:53.929299 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1415/2851] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:43:53 2022] POST /identity/v3/projects => generated 375 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:43:53.933509 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d124d0b-fc22-46c0-9628-eeb20f07cb0f admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:53.943577 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d124d0b-fc22-46c0-9628-eeb20f07cb0f admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:53.999319 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6b42eb-d782-49cc-b953-e4bfb6ad1b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:53.999927 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6b42eb-d782-49cc-b953-e4bfb6ad1b05 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:54.000247 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6b42eb-d782-49cc-b953-e4bfb6ad1b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:54.000525 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6b42eb-d782-49cc-b953-e4bfb6ad1b05 admin admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:54.001188 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6b42eb-d782-49cc-b953-e4bfb6ad1b05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:54.003088 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6b42eb-d782-49cc-b953-e4bfb6ad1b05 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:54.105237 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1437/2852] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:43:53 2022] POST /identity/v3/users => generated 288 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:43:54.109344 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38df548-c885-4563-a5b9-0f0e69dc5c9f admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:54.119683 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38df548-c885-4563-a5b9-0f0e69dc5c9f admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:54.124803 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef183c64-c6e2-404a-a467-c21a8f4218dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:54.125421 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef183c64-c6e2-404a-a467-c21a8f4218dc admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:54.125708 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef183c64-c6e2-404a-a467-c21a8f4218dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:54.132235 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef183c64-c6e2-404a-a467-c21a8f4218dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:54.132235 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef183c64-c6e2-404a-a467-c21a8f4218dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:54.132235 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef183c64-c6e2-404a-a467-c21a8f4218dc admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:54.138399 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1416/2853] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:43:54.142613 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6b42eb-d782-49cc-b953-e4bfb6ad1b05 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:54.153685 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6b42eb-d782-49cc-b953-e4bfb6ad1b05 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:54.159392 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc319dd-c520-453c-bf50-f20ed0b1787a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:54.160830 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc319dd-c520-453c-bf50-f20ed0b1787a admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:54.160830 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc319dd-c520-453c-bf50-f20ed0b1787a admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:54.160830 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc319dd-c520-453c-bf50-f20ed0b1787a admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:54.161286 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc319dd-c520-453c-bf50-f20ed0b1787a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:54.162889 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc319dd-c520-453c-bf50-f20ed0b1787a admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:54.175821 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1438/2854] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:43:54.180028 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef183c64-c6e2-404a-a467-c21a8f4218dc admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:54.190724 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef183c64-c6e2-404a-a467-c21a8f4218dc admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:54.194949 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce6e37c-6817-4297-bf5f-38d483732c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:54.195602 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6e37c-6817-4297-bf5f-38d483732c9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:54.195889 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6e37c-6817-4297-bf5f-38d483732c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:54.196204 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6e37c-6817-4297-bf5f-38d483732c9f admin admin] PATH_INFO: `/v3/projects/527bac41a3b3426fb949b34fb7b85f31/users/044551282f8b491ea86181163f2d46df/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:54.211486 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce6e37c-6817-4297-bf5f-38d483732c9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=527bac41a3b3426fb949b34fb7b85f31, user_id=044551282f8b491ea86181163f2d46df, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:54.214323 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce6e37c-6817-4297-bf5f-38d483732c9f admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:54.284617 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1417/2855] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:43:54 2022] PUT /identity/v3/projects/527bac41a3b3426fb949b34fb7b85f31/users/044551282f8b491ea86181163f2d46df/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:43:54.295767 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e27785-deb9-4e10-919b-d1a8840ff6bb None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:54.296058 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e27785-deb9-4e10-919b-d1a8840ff6bb None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:54.296374 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e27785-deb9-4e10-919b-d1a8840ff6bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:54.323223 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-14e27785-deb9-4e10-919b-d1a8840ff6bb None None] MFA Rules not processed for user `044551282f8b491ea86181163f2d46df`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:43:54.324467 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-14e27785-deb9-4e10-919b-d1a8840ff6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:54.386683 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1439/2856] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:43:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:43:54.391967 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce6e37c-6817-4297-bf5f-38d483732c9f admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:54.440005 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:43:54.461521 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce6e37c-6817-4297-bf5f-38d483732c9f admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:54.466282 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c26e2-a942-424e-9739-91f65b5ab896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:54.466889 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c26e2-a942-424e-9739-91f65b5ab896 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:54.467171 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c26e2-a942-424e-9739-91f65b5ab896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:54.467473 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c26e2-a942-424e-9739-91f65b5ab896 admin admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:54.468153 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06c26e2-a942-424e-9739-91f65b5ab896 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:54.470053 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06c26e2-a942-424e-9739-91f65b5ab896 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:54.563382 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1418/2857] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:43:54 2022] POST /identity/v3/projects => generated 375 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:43:54.567455 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e27785-deb9-4e10-919b-d1a8840ff6bb None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:54.578674 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e27785-deb9-4e10-919b-d1a8840ff6bb None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:54.633235 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773e0c2-17e5-4263-a963-4188a258cdb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:54.633870 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773e0c2-17e5-4263-a963-4188a258cdb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:54.634153 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773e0c2-17e5-4263-a963-4188a258cdb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:54.634933 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773e0c2-17e5-4263-a963-4188a258cdb5 admin admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:54.635165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3773e0c2-17e5-4263-a963-4188a258cdb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:54.637067 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3773e0c2-17e5-4263-a963-4188a258cdb5 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:54.647301 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:43:54.765128 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1440/2858] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:43:54 2022] POST /identity/v3/users => generated 288 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 15:43:54.768851 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c26e2-a942-424e-9739-91f65b5ab896 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:54.779156 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c26e2-a942-424e-9739-91f65b5ab896 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:54.783885 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6efaf-b5a2-41b7-99d0-c1e2a15ab354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:54.784523 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6efaf-b5a2-41b7-99d0-c1e2a15ab354 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:54.784806 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6efaf-b5a2-41b7-99d0-c1e2a15ab354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:54.785081 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6efaf-b5a2-41b7-99d0-c1e2a15ab354 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:54.785766 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c6efaf-b5a2-41b7-99d0-c1e2a15ab354 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:54.787343 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c6efaf-b5a2-41b7-99d0-c1e2a15ab354 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:54.797014 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1419/2859] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:43:54.800834 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773e0c2-17e5-4263-a963-4188a258cdb5 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:54.810742 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773e0c2-17e5-4263-a963-4188a258cdb5 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:54.815929 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c263912-ee17-4817-92ed-b4b69b4af355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:54.816559 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c263912-ee17-4817-92ed-b4b69b4af355 admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:54.816845 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c263912-ee17-4817-92ed-b4b69b4af355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:54.817126 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c263912-ee17-4817-92ed-b4b69b4af355 admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:54.817814 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c263912-ee17-4817-92ed-b4b69b4af355 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:54.819405 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c263912-ee17-4817-92ed-b4b69b4af355 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:54.829857 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1441/2860] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:43:54.833781 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6efaf-b5a2-41b7-99d0-c1e2a15ab354 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:54.843439 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6efaf-b5a2-41b7-99d0-c1e2a15ab354 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:54.847893 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c008b0-beee-4cc1-8e48-f4bcd86d3e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:54.848563 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c008b0-beee-4cc1-8e48-f4bcd86d3e54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:54.848849 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c008b0-beee-4cc1-8e48-f4bcd86d3e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:54.849127 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c008b0-beee-4cc1-8e48-f4bcd86d3e54 admin admin] PATH_INFO: `/v3/projects/43c57992731546b9bc4d5efd6c97056f/users/9783b2a55ca940869d6386e1a02eb9b0/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:54.865016 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c008b0-beee-4cc1-8e48-f4bcd86d3e54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43c57992731546b9bc4d5efd6c97056f, user_id=9783b2a55ca940869d6386e1a02eb9b0, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:54.867640 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c008b0-beee-4cc1-8e48-f4bcd86d3e54 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:54.918173 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1420/2861] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:43:54 2022] PUT /identity/v3/projects/43c57992731546b9bc4d5efd6c97056f/users/9783b2a55ca940869d6386e1a02eb9b0/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:43:54.930205 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97271e38-4522-434b-a295-0ec3dbc86756 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:54.930522 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97271e38-4522-434b-a295-0ec3dbc86756 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:54.930808 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97271e38-4522-434b-a295-0ec3dbc86756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:54.959938 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-97271e38-4522-434b-a295-0ec3dbc86756 None None] MFA Rules not processed for user `9783b2a55ca940869d6386e1a02eb9b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:43:54.961235 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-97271e38-4522-434b-a295-0ec3dbc86756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:55.024995 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1442/2862] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:43:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:43:55.029505 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c008b0-beee-4cc1-8e48-f4bcd86d3e54 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:55.091675 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c008b0-beee-4cc1-8e48-f4bcd86d3e54 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:55.097414 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9390c-2946-4245-bbb1-545ddf59bfe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:55.098031 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9390c-2946-4245-bbb1-545ddf59bfe3 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:55.098321 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9390c-2946-4245-bbb1-545ddf59bfe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:55.098600 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9390c-2946-4245-bbb1-545ddf59bfe3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:55.099265 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d9390c-2946-4245-bbb1-545ddf59bfe3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:55.101234 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d9390c-2946-4245-bbb1-545ddf59bfe3 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:55.182269 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1421/2863] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:43:55 2022] POST /identity/v3/projects => generated 375 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:43:55.187134 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97271e38-4522-434b-a295-0ec3dbc86756 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:55.198104 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97271e38-4522-434b-a295-0ec3dbc86756 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:55.254448 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66231514-90a4-4142-a79d-530e0981bfec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:55.255065 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66231514-90a4-4142-a79d-530e0981bfec admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:55.255349 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66231514-90a4-4142-a79d-530e0981bfec admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:55.255626 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66231514-90a4-4142-a79d-530e0981bfec admin admin] PATH_INFO: `/v3/users` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:55.256311 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66231514-90a4-4142-a79d-530e0981bfec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:55.258230 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66231514-90a4-4142-a79d-530e0981bfec admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:55.369416 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1443/2864] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:43:55 2022] POST /identity/v3/users => generated 294 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:43:55.373433 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9390c-2946-4245-bbb1-545ddf59bfe3 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:55.384273 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9390c-2946-4245-bbb1-545ddf59bfe3 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:55.389227 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8079e2-174a-4b35-a1e7-8c3ed7fcf4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:55.390485 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8079e2-174a-4b35-a1e7-8c3ed7fcf4f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:55.390773 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8079e2-174a-4b35-a1e7-8c3ed7fcf4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:55.391053 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8079e2-174a-4b35-a1e7-8c3ed7fcf4f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:55.391749 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8079e2-174a-4b35-a1e7-8c3ed7fcf4f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:55.393391 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8079e2-174a-4b35-a1e7-8c3ed7fcf4f0 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:55.404019 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1422/2865] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:43:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:43:55.408371 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66231514-90a4-4142-a79d-530e0981bfec admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:55.418360 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66231514-90a4-4142-a79d-530e0981bfec admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:55.422978 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a5e41-6cbb-412b-97c6-ee67b7f38ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:55.423571 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a5e41-6cbb-412b-97c6-ee67b7f38ed7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:55.423855 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a5e41-6cbb-412b-97c6-ee67b7f38ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:55.424161 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a5e41-6cbb-412b-97c6-ee67b7f38ed7 admin admin] PATH_INFO: `/v3/projects/ab9a6cf544b746a8bb3db62b94a5d69d/users/9932db55f0e144899d0de98ac138362f/roles/7c88c815db854954b516ab97cd471647` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:55.439148 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90a5e41-6cbb-412b-97c6-ee67b7f38ed7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab9a6cf544b746a8bb3db62b94a5d69d, user_id=9932db55f0e144899d0de98ac138362f, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:55.441809 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90a5e41-6cbb-412b-97c6-ee67b7f38ed7 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:55.512234 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1444/2866] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:43:55 2022] PUT /identity/v3/projects/ab9a6cf544b746a8bb3db62b94a5d69d/users/9932db55f0e144899d0de98ac138362f/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:43:55.516729 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8079e2-174a-4b35-a1e7-8c3ed7fcf4f0 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:55.526618 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8079e2-174a-4b35-a1e7-8c3ed7fcf4f0 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:55.581151 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669be254-5ca3-4949-aed2-9314ce6009e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:55.581767 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669be254-5ca3-4949-aed2-9314ce6009e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:55.582053 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669be254-5ca3-4949-aed2-9314ce6009e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:55.582334 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669be254-5ca3-4949-aed2-9314ce6009e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:55.583029 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669be254-5ca3-4949-aed2-9314ce6009e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:55.584650 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669be254-5ca3-4949-aed2-9314ce6009e5 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:55.589686 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:43:55.594648 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1423/2867] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:43:55 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:43:55.598755 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a5e41-6cbb-412b-97c6-ee67b7f38ed7 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:55.609651 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a5e41-6cbb-412b-97c6-ee67b7f38ed7 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:55.614266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b4068-fd56-4647-aa0e-3774650a3f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:55.614927 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b4068-fd56-4647-aa0e-3774650a3f00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:55.615252 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b4068-fd56-4647-aa0e-3774650a3f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:55.615548 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b4068-fd56-4647-aa0e-3774650a3f00 admin admin] PATH_INFO: `/v3/projects/ab9a6cf544b746a8bb3db62b94a5d69d/users/9932db55f0e144899d0de98ac138362f/roles/7c88c815db854954b516ab97cd471647` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:55.617568 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370b4068-fd56-4647-aa0e-3774650a3f00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab9a6cf544b746a8bb3db62b94a5d69d, user_id=9932db55f0e144899d0de98ac138362f, role_id=7c88c815db854954b516ab97cd471647)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:55.620287 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370b4068-fd56-4647-aa0e-3774650a3f00 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:55.630860 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1445/2868] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:43:55 2022] PUT /identity/v3/projects/ab9a6cf544b746a8bb3db62b94a5d69d/users/9932db55f0e144899d0de98ac138362f/roles/7c88c815db854954b516ab97cd471647 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:43:55.641965 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35544b-f3f1-4131-9390-19f6804f24e1 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:55.642260 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35544b-f3f1-4131-9390-19f6804f24e1 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:55.642545 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35544b-f3f1-4131-9390-19f6804f24e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:55.670996 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-9c35544b-f3f1-4131-9390-19f6804f24e1 None None] MFA Rules not processed for user `9932db55f0e144899d0de98ac138362f`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:43:55.672260 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-9c35544b-f3f1-4131-9390-19f6804f24e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:55.740407 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1424/2869] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:43:55 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:43:55.752503 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b4068-fd56-4647-aa0e-3774650a3f00 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:55.754215 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-370b4068-fd56-4647-aa0e-3774650a3f00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:55.781444 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:43:55.810449 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b4068-fd56-4647-aa0e-3774650a3f00 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:55.814874 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c882cb-a0bb-4c39-a3cb-563d9be18724 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:55.815510 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c882cb-a0bb-4c39-a3cb-563d9be18724 service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:55.815951 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c882cb-a0bb-4c39-a3cb-563d9be18724 service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:55.816299 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c882cb-a0bb-4c39-a3cb-563d9be18724 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:55.871355 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c882cb-a0bb-4c39-a3cb-563d9be18724 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:55.872836 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c882cb-a0bb-4c39-a3cb-563d9be18724 service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:55.882347 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1446/2870] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:43:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:43:57.757872 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35544b-f3f1-4131-9390-19f6804f24e1 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:57.759257 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-9c35544b-f3f1-4131-9390-19f6804f24e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:43:57.771530 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35544b-f3f1-4131-9390-19f6804f24e1 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:57.775231 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a920c-885d-4ae4-955f-8eb88cdb26b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:57.775841 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a920c-885d-4ae4-955f-8eb88cdb26b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:57.776166 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a920c-885d-4ae4-955f-8eb88cdb26b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:57.776448 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a920c-885d-4ae4-955f-8eb88cdb26b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:57.832457 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8a920c-885d-4ae4-955f-8eb88cdb26b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:57.833885 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8a920c-885d-4ae4-955f-8eb88cdb26b2 service cinder] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:57.843235 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1425/2871] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:43:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:43:59.106165 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c882cb-a0bb-4c39-a3cb-563d9be18724 service cinder] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:43:59.115365 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c882cb-a0bb-4c39-a3cb-563d9be18724 service cinder] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:43:59.118970 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d334aaa-fa80-49af-a409-82dabde921b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:43:59.119577 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d334aaa-fa80-49af-a409-82dabde921b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:43:59.119885 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d334aaa-fa80-49af-a409-82dabde921b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:43:59.120196 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d334aaa-fa80-49af-a409-82dabde921b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:43:59.129551 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d334aaa-fa80-49af-a409-82dabde921b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:43:59.131156 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d334aaa-fa80-49af-a409-82dabde921b2 service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:43:59.141011 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1447/2872] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:43:59 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:44:03.871950 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a920c-885d-4ae4-955f-8eb88cdb26b2 service cinder] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:44:03.934499 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a920c-885d-4ae4-955f-8eb88cdb26b2 service cinder] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:44:03.939121 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d64154e-1b8e-443d-8cab-2afd089d6bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:44:03.939751 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64154e-1b8e-443d-8cab-2afd089d6bf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:44:03.940031 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64154e-1b8e-443d-8cab-2afd089d6bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:44:03.940337 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64154e-1b8e-443d-8cab-2afd089d6bf0 admin admin] PATH_INFO: `/v3/users/044551282f8b491ea86181163f2d46df` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:44:03.941602 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d64154e-1b8e-443d-8cab-2afd089d6bf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=044551282f8b491ea86181163f2d46df)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:44:03.943695 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d64154e-1b8e-443d-8cab-2afd089d6bf0 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:44:04.054120 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:44:04.159434 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-2d64154e-1b8e-443d-8cab-2afd089d6bf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '044551282f8b491ea86181163f2d46df'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:04.166858 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-2d64154e-1b8e-443d-8cab-2afd089d6bf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '044551282f8b491ea86181163f2d46df'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:04.167658 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-2d64154e-1b8e-443d-8cab-2afd089d6bf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '044551282f8b491ea86181163f2d46df'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:04.179759 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-2d64154e-1b8e-443d-8cab-2afd089d6bf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '044551282f8b491ea86181163f2d46df'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:04.180457 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-2d64154e-1b8e-443d-8cab-2afd089d6bf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '044551282f8b491ea86181163f2d46df'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:04.258146 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1426/2873] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:44:03 2022] DELETE /identity/v3/users/044551282f8b491ea86181163f2d46df => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:44:04.262587 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674d568-d9ae-4dbe-bb1c-a1e3505a7b10 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:44:04.263134 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674d568-d9ae-4dbe-bb1c-a1e3505a7b10 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:44:04.263285 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674d568-d9ae-4dbe-bb1c-a1e3505a7b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:44:04.282910 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-3674d568-d9ae-4dbe-bb1c-a1e3505a7b10 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:44:04.284409 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-3674d568-d9ae-4dbe-bb1c-a1e3505a7b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:44:04.342334 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1448/2874] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:44:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:44:04.351832 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d64154e-1b8e-443d-8cab-2afd089d6bf0 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:44:04.353271 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-2d64154e-1b8e-443d-8cab-2afd089d6bf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:44:04.414190 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d64154e-1b8e-443d-8cab-2afd089d6bf0 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:44:04.417804 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7c841-48f8-4954-b2d7-81a4267abec6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:44:04.418402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7c841-48f8-4954-b2d7-81a4267abec6 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:44:04.418682 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7c841-48f8-4954-b2d7-81a4267abec6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:44:04.418956 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7c841-48f8-4954-b2d7-81a4267abec6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:44:04.427743 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede7c841-48f8-4954-b2d7-81a4267abec6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:44:04.429193 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede7c841-48f8-4954-b2d7-81a4267abec6 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:44:04.439229 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1427/2875] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:44:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:44:04.873991 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3674d568-d9ae-4dbe-bb1c-a1e3505a7b10 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:44:04.876175 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-3674d568-d9ae-4dbe-bb1c-a1e3505a7b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:44:04.889947 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3674d568-d9ae-4dbe-bb1c-a1e3505a7b10 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:44:04.895482 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfec05cf-fc63-4159-964d-0904532725c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:44:04.896176 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec05cf-fc63-4159-964d-0904532725c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:44:04.896466 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec05cf-fc63-4159-964d-0904532725c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:44:04.896931 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec05cf-fc63-4159-964d-0904532725c2 admin admin] PATH_INFO: `/v3/projects/527bac41a3b3426fb949b34fb7b85f31` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:44:04.898130 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfec05cf-fc63-4159-964d-0904532725c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=527bac41a3b3426fb949b34fb7b85f31)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:44:04.900540 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfec05cf-fc63-4159-964d-0904532725c2 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:44:05.030607 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-bfec05cf-fc63-4159-964d-0904532725c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '527bac41a3b3426fb949b34fb7b85f31', 'request_id': 'req-bfec05cf-fc63-4159-964d-0904532725c2'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:05.037692 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-bfec05cf-fc63-4159-964d-0904532725c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '527bac41a3b3426fb949b34fb7b85f31', 'request_id': 'req-bfec05cf-fc63-4159-964d-0904532725c2'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:05.042399 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:44:05.043598 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-bfec05cf-fc63-4159-964d-0904532725c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '527bac41a3b3426fb949b34fb7b85f31', 'request_id': 'req-bfec05cf-fc63-4159-964d-0904532725c2'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:05.134786 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-bfec05cf-fc63-4159-964d-0904532725c2 admin admin] The token cache is being invalidate because project 527bac41a3b3426fb949b34fb7b85f31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:44:05.135216 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-bfec05cf-fc63-4159-964d-0904532725c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:05.135844 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-bfec05cf-fc63-4159-964d-0904532725c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:05.137380 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1449/2876] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:44:04 2022] DELETE /identity/v3/projects/527bac41a3b3426fb949b34fb7b85f31 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:44:05.141555 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7c841-48f8-4954-b2d7-81a4267abec6 service neutron] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:44:05.151436 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7c841-48f8-4954-b2d7-81a4267abec6 service neutron] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:44:05.215704 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb24da95-d9bd-4a2f-8ade-611e31310f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:44:05.216391 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb24da95-d9bd-4a2f-8ade-611e31310f49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:44:05.216678 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb24da95-d9bd-4a2f-8ade-611e31310f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:44:05.216955 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb24da95-d9bd-4a2f-8ade-611e31310f49 admin admin] PATH_INFO: `/v3/users/9783b2a55ca940869d6386e1a02eb9b0` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:44:05.220033 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb24da95-d9bd-4a2f-8ade-611e31310f49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9783b2a55ca940869d6386e1a02eb9b0)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:44:05.220361 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfec05cf-fc63-4159-964d-0904532725c2 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:44:05.221771 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-bfec05cf-fc63-4159-964d-0904532725c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:44:05.222030 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb24da95-d9bd-4a2f-8ade-611e31310f49 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:44:05.284487 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfec05cf-fc63-4159-964d-0904532725c2 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:44:05.288206 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f6597-6a4d-41fb-aabf-4c005b163fa3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:44:05.288828 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f6597-6a4d-41fb-aabf-4c005b163fa3 service neutron] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:44:05.289117 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f6597-6a4d-41fb-aabf-4c005b163fa3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:44:05.289398 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f6597-6a4d-41fb-aabf-4c005b163fa3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:44:05.291175 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-919f6597-6a4d-41fb-aabf-4c005b163fa3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:44:05.303948 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919f6597-6a4d-41fb-aabf-4c005b163fa3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:44:05.305454 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919f6597-6a4d-41fb-aabf-4c005b163fa3 service neutron] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:44:05.315052 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1450/2877] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:44:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 15:44:05.411617 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-eb24da95-d9bd-4a2f-8ade-611e31310f49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9783b2a55ca940869d6386e1a02eb9b0'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:05.413010 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:44:05.419259 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-eb24da95-d9bd-4a2f-8ade-611e31310f49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9783b2a55ca940869d6386e1a02eb9b0'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:05.419946 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-eb24da95-d9bd-4a2f-8ade-611e31310f49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9783b2a55ca940869d6386e1a02eb9b0'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:05.432108 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-eb24da95-d9bd-4a2f-8ade-611e31310f49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9783b2a55ca940869d6386e1a02eb9b0'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:05.432792 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-eb24da95-d9bd-4a2f-8ade-611e31310f49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9783b2a55ca940869d6386e1a02eb9b0'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:05.522834 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1428/2878] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:44:05 2022] DELETE /identity/v3/users/9783b2a55ca940869d6386e1a02eb9b0 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:44:05.982362 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f6597-6a4d-41fb-aabf-4c005b163fa3 service neutron] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:44:05.983993 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-919f6597-6a4d-41fb-aabf-4c005b163fa3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:44:06.004273 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f6597-6a4d-41fb-aabf-4c005b163fa3 service neutron] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:44:06.070732 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1699522-b8d8-45ad-976b-8ce28498f011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:44:06.071770 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1699522-b8d8-45ad-976b-8ce28498f011 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:44:06.072250 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1699522-b8d8-45ad-976b-8ce28498f011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:44:06.072765 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1699522-b8d8-45ad-976b-8ce28498f011 admin admin] PATH_INFO: `/v3/projects/43c57992731546b9bc4d5efd6c97056f` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:44:06.073908 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1699522-b8d8-45ad-976b-8ce28498f011 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43c57992731546b9bc4d5efd6c97056f)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:44:06.075814 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1699522-b8d8-45ad-976b-8ce28498f011 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:44:06.205077 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-d1699522-b8d8-45ad-976b-8ce28498f011 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43c57992731546b9bc4d5efd6c97056f', 'request_id': 'req-d1699522-b8d8-45ad-976b-8ce28498f011'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:06.211266 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-d1699522-b8d8-45ad-976b-8ce28498f011 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43c57992731546b9bc4d5efd6c97056f', 'request_id': 'req-d1699522-b8d8-45ad-976b-8ce28498f011'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:06.217702 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-d1699522-b8d8-45ad-976b-8ce28498f011 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43c57992731546b9bc4d5efd6c97056f', 'request_id': 'req-d1699522-b8d8-45ad-976b-8ce28498f011'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:06.283945 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-d1699522-b8d8-45ad-976b-8ce28498f011 admin admin] The token cache is being invalidate because project 43c57992731546b9bc4d5efd6c97056f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:44:06.284357 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-d1699522-b8d8-45ad-976b-8ce28498f011 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:06.288522 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-d1699522-b8d8-45ad-976b-8ce28498f011 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:06.290806 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1451/2879] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:44:05 2022] DELETE /identity/v3/projects/43c57992731546b9bc4d5efd6c97056f => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:44:06.295527 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb24da95-d9bd-4a2f-8ade-611e31310f49 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:44:06.298736 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-eb24da95-d9bd-4a2f-8ade-611e31310f49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:44:06.316003 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb24da95-d9bd-4a2f-8ade-611e31310f49 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:44:06.386488 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254f2bf-d16b-45fd-a2bc-449e914ff354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:44:06.387458 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254f2bf-d16b-45fd-a2bc-449e914ff354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:44:06.387926 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254f2bf-d16b-45fd-a2bc-449e914ff354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:44:06.388280 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254f2bf-d16b-45fd-a2bc-449e914ff354 admin admin] PATH_INFO: `/v3/users/9932db55f0e144899d0de98ac138362f` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:44:06.389499 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5254f2bf-d16b-45fd-a2bc-449e914ff354 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9932db55f0e144899d0de98ac138362f)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:44:06.392368 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5254f2bf-d16b-45fd-a2bc-449e914ff354 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:44:06.581937 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-5254f2bf-d16b-45fd-a2bc-449e914ff354 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9932db55f0e144899d0de98ac138362f'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:06.588819 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-5254f2bf-d16b-45fd-a2bc-449e914ff354 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9932db55f0e144899d0de98ac138362f'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:06.589503 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-5254f2bf-d16b-45fd-a2bc-449e914ff354 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9932db55f0e144899d0de98ac138362f'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:06.601396 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-5254f2bf-d16b-45fd-a2bc-449e914ff354 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9932db55f0e144899d0de98ac138362f'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:06.602057 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-5254f2bf-d16b-45fd-a2bc-449e914ff354 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9932db55f0e144899d0de98ac138362f'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:06.676157 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1429/2880] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:44:06 2022] DELETE /identity/v3/users/9932db55f0e144899d0de98ac138362f => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 15:44:07.193405 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1699522-b8d8-45ad-976b-8ce28498f011 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:44:07.194910 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-d1699522-b8d8-45ad-976b-8ce28498f011 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:44:07.230285 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1699522-b8d8-45ad-976b-8ce28498f011 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:44:07.259377 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:44:07.300351 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74430772-c538-42df-9ff3-0382c5a3cf40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:44:07.302264 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74430772-c538-42df-9ff3-0382c5a3cf40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:44:07.302264 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74430772-c538-42df-9ff3-0382c5a3cf40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:44:07.302264 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74430772-c538-42df-9ff3-0382c5a3cf40 admin admin] PATH_INFO: `/v3/projects/ab9a6cf544b746a8bb3db62b94a5d69d` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:44:07.302829 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74430772-c538-42df-9ff3-0382c5a3cf40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab9a6cf544b746a8bb3db62b94a5d69d)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:44:07.304955 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74430772-c538-42df-9ff3-0382c5a3cf40 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:44:07.441874 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-74430772-c538-42df-9ff3-0382c5a3cf40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab9a6cf544b746a8bb3db62b94a5d69d', 'request_id': 'req-74430772-c538-42df-9ff3-0382c5a3cf40'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:07.448282 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-74430772-c538-42df-9ff3-0382c5a3cf40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab9a6cf544b746a8bb3db62b94a5d69d', 'request_id': 'req-74430772-c538-42df-9ff3-0382c5a3cf40'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:07.454183 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-74430772-c538-42df-9ff3-0382c5a3cf40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab9a6cf544b746a8bb3db62b94a5d69d', 'request_id': 'req-74430772-c538-42df-9ff3-0382c5a3cf40'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:07.544655 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-74430772-c538-42df-9ff3-0382c5a3cf40 admin admin] The token cache is being invalidate because project ab9a6cf544b746a8bb3db62b94a5d69d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:44:07.549040 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-74430772-c538-42df-9ff3-0382c5a3cf40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:07.549040 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-74430772-c538-42df-9ff3-0382c5a3cf40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:07.549040 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1452/2881] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:44:07 2022] DELETE /identity/v3/projects/ab9a6cf544b746a8bb3db62b94a5d69d => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:44:07.552569 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b3bb1-97a8-4618-8486-6cce104346b6 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:44:07.553101 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b3bb1-97a8-4618-8486-6cce104346b6 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:44:07.553312 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b3bb1-97a8-4618-8486-6cce104346b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:44:07.574167 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-479b3bb1-97a8-4618-8486-6cce104346b6 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:44:07.575506 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-479b3bb1-97a8-4618-8486-6cce104346b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:44:07.618419 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:44:07.639350 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1430/2882] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:44:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:44:07.647327 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded3856-009f-4ed7-a075-460de5754665 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:44:07.658373 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded3856-009f-4ed7-a075-460de5754665 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:44:07.658373 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded3856-009f-4ed7-a075-460de5754665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:44:07.667506 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-cded3856-009f-4ed7-a075-460de5754665 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:44:07.668761 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-cded3856-009f-4ed7-a075-460de5754665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:44:07.677545 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1453/2883] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:44:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 15:44:07.681910 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b3bb1-97a8-4618-8486-6cce104346b6 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:44:07.692574 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b3bb1-97a8-4618-8486-6cce104346b6 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:44:07.697497 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d096059-753e-4072-9e41-24017f300c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:44:07.698126 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d096059-753e-4072-9e41-24017f300c66 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:44:07.698415 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d096059-753e-4072-9e41-24017f300c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:44:07.698691 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d096059-753e-4072-9e41-24017f300c66 admin admin] PATH_INFO: `/v3/domains` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:44:07.699398 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d096059-753e-4072-9e41-24017f300c66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80541) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 15:44:07.699710 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d096059-753e-4072-9e41-24017f300c66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:44:07.701383 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d096059-753e-4072-9e41-24017f300c66 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:44:07.714109 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1431/2884] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 5 15:44:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 15:44:07.718615 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded3856-009f-4ed7-a075-460de5754665 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:44:07.728641 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded3856-009f-4ed7-a075-460de5754665 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:44:07.751091 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2734d57-641a-4d52-a7c9-48edc0d41927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:44:07.751091 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2734d57-641a-4d52-a7c9-48edc0d41927 admin admin] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:44:07.751091 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2734d57-641a-4d52-a7c9-48edc0d41927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:44:07.751091 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2734d57-641a-4d52-a7c9-48edc0d41927 admin admin] PATH_INFO: `/v3/projects` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:44:07.751479 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2734d57-641a-4d52-a7c9-48edc0d41927 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:44:07.751479 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2734d57-641a-4d52-a7c9-48edc0d41927 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:44:07.879806 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1454/2885] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 5 15:44:07 2022] POST /identity/v3/projects => generated 385 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:44:07.883835 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d096059-753e-4072-9e41-24017f300c66 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:44:07.903564 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d096059-753e-4072-9e41-24017f300c66 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:44:07.985429 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11da0e-c74e-4b5e-8797-6e57fd9f0920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:44:07.986194 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11da0e-c74e-4b5e-8797-6e57fd9f0920 admin admin] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:44:07.986574 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11da0e-c74e-4b5e-8797-6e57fd9f0920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:44:07.986947 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11da0e-c74e-4b5e-8797-6e57fd9f0920 admin admin] PATH_INFO: `/v3/users` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:44:07.987705 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b11da0e-c74e-4b5e-8797-6e57fd9f0920 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:44:07.989910 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b11da0e-c74e-4b5e-8797-6e57fd9f0920 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:44:08.055591 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1432/2886] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 5 15:44:07 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 15:44:08.059691 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2734d57-641a-4d52-a7c9-48edc0d41927 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:44:08.071039 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2734d57-641a-4d52-a7c9-48edc0d41927 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:44:08.075793 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9684a0-530b-4ff0-8a87-31e5c0b65d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:44:08.076492 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9684a0-530b-4ff0-8a87-31e5c0b65d9d admin admin] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:44:08.076783 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9684a0-530b-4ff0-8a87-31e5c0b65d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:44:08.077064 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9684a0-530b-4ff0-8a87-31e5c0b65d9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:44:08.077764 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9684a0-530b-4ff0-8a87-31e5c0b65d9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:44:08.080101 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9684a0-530b-4ff0-8a87-31e5c0b65d9d admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:44:08.090114 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1455/2887] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 15:44:08.094032 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11da0e-c74e-4b5e-8797-6e57fd9f0920 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:44:08.104172 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11da0e-c74e-4b5e-8797-6e57fd9f0920 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:44:08.108857 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6cff4e-b58d-4271-b6a5-8b0f46f855a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:44:08.109489 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6cff4e-b58d-4271-b6a5-8b0f46f855a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:44:08.109774 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6cff4e-b58d-4271-b6a5-8b0f46f855a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:44:08.110053 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6cff4e-b58d-4271-b6a5-8b0f46f855a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:44:08.110747 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6cff4e-b58d-4271-b6a5-8b0f46f855a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:44:08.112364 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6cff4e-b58d-4271-b6a5-8b0f46f855a5 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:44:08.123011 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1433/2888] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 5 15:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 15:44:08.128221 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9684a0-530b-4ff0-8a87-31e5c0b65d9d admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:44:08.142374 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9684a0-530b-4ff0-8a87-31e5c0b65d9d admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:44:08.148281 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d08e81e-ca07-4d7d-b880-245b94066e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:44:08.148986 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08e81e-ca07-4d7d-b880-245b94066e9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:44:08.149269 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08e81e-ca07-4d7d-b880-245b94066e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:44:08.151407 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08e81e-ca07-4d7d-b880-245b94066e9b admin admin] PATH_INFO: `/v3/projects/6b22abde51b34b90a6601f1dbfda6f3e/users/38765e7855544fcd956f367e42e461e5/roles/d6c91838e4944cfbb23e1b6ecb009d26` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:44:08.177059 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d08e81e-ca07-4d7d-b880-245b94066e9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b22abde51b34b90a6601f1dbfda6f3e, user_id=38765e7855544fcd956f367e42e461e5, role_id=d6c91838e4944cfbb23e1b6ecb009d26)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:44:08.180688 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d08e81e-ca07-4d7d-b880-245b94066e9b admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:44:08.278658 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1456/2889] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 5 15:44:08 2022] PUT /identity/v3/projects/6b22abde51b34b90a6601f1dbfda6f3e/users/38765e7855544fcd956f367e42e461e5/roles/d6c91838e4944cfbb23e1b6ecb009d26 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:44:08.289402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4fa66-8470-44f1-b9ec-f73d851e9403 None None] REQUEST_METHOD: `POST` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:44:08.289692 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4fa66-8470-44f1-b9ec-f73d851e9403 None None] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:44:08.289979 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4fa66-8470-44f1-b9ec-f73d851e9403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:44:08.321996 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.auth.core [None req-def4fa66-8470-44f1-b9ec-f73d851e9403 None None] MFA Rules not processed for user `38765e7855544fcd956f367e42e461e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80541) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:44:08.323306 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-def4fa66-8470-44f1-b9ec-f73d851e9403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:44:08.391754 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1434/2890] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:44:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:44:08.403227 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d08e81e-ca07-4d7d-b880-245b94066e9b admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:44:08.405034 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-9d08e81e-ca07-4d7d-b880-245b94066e9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:44:08.472527 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:44:08.477137 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d08e81e-ca07-4d7d-b880-245b94066e9b admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:44:08.481126 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6923c1a-e6a2-42fb-a0b0-bb3a0600c64f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e974f8d9a06d489884a3b00b95dd88e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:44:08.481765 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6923c1a-e6a2-42fb-a0b0-bb3a0600c64f service cinder] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:44:08.482058 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6923c1a-e6a2-42fb-a0b0-bb3a0600c64f service cinder] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:44:08.482341 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6923c1a-e6a2-42fb-a0b0-bb3a0600c64f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:44:08.497625 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6923c1a-e6a2-42fb-a0b0-bb3a0600c64f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:44:08.499505 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6923c1a-e6a2-42fb-a0b0-bb3a0600c64f service cinder] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:44:08.513180 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1457/2891] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:44:08 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:44:11.622722 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def4fa66-8470-44f1-b9ec-f73d851e9403 None None] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:44:11.666913 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:44:11.703743 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def4fa66-8470-44f1-b9ec-f73d851e9403 None None] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:44:11.715168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86b68b-0111-4e91-b82a-d4cbdebf5de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:44:11.715168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86b68b-0111-4e91-b82a-d4cbdebf5de1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:44:11.715168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86b68b-0111-4e91-b82a-d4cbdebf5de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:44:11.715168 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86b68b-0111-4e91-b82a-d4cbdebf5de1 admin admin] PATH_INFO: `/v3/users/38765e7855544fcd956f367e42e461e5` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:44:11.715561 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b86b68b-0111-4e91-b82a-d4cbdebf5de1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38765e7855544fcd956f367e42e461e5)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:44:11.715799 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b86b68b-0111-4e91-b82a-d4cbdebf5de1 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:44:11.909876 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-2b86b68b-0111-4e91-b82a-d4cbdebf5de1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38765e7855544fcd956f367e42e461e5'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:11.917010 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-2b86b68b-0111-4e91-b82a-d4cbdebf5de1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38765e7855544fcd956f367e42e461e5'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:11.917716 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-2b86b68b-0111-4e91-b82a-d4cbdebf5de1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38765e7855544fcd956f367e42e461e5'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:11.931070 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-2b86b68b-0111-4e91-b82a-d4cbdebf5de1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38765e7855544fcd956f367e42e461e5'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:11.931740 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-2b86b68b-0111-4e91-b82a-d4cbdebf5de1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38765e7855544fcd956f367e42e461e5'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:12.012929 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1435/2892] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:44:11 2022] DELETE /identity/v3/users/38765e7855544fcd956f367e42e461e5 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:44:12.018095 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1a035-3bfb-40a8-b4f6-0625ca009220 None None] REQUEST_METHOD: `POST` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:44:12.018590 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1a035-3bfb-40a8-b4f6-0625ca009220 None None] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:44:12.018897 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1a035-3bfb-40a8-b4f6-0625ca009220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:44:12.038956 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.auth.core [None req-d9e1a035-3bfb-40a8-b4f6-0625ca009220 None None] MFA Rules not processed for user `338271833df8486b9ef50964b5ec7797`. Rule list: `[]` (Enabled: `True`). {{(pid=80542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 15:44:12.040151 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-d9e1a035-3bfb-40a8-b4f6-0625ca009220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:44:12.104032 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1458/2893] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 5 15:44:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 15:44:12.115449 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86b68b-0111-4e91-b82a-d4cbdebf5de1 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:44:12.117108 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-2b86b68b-0111-4e91-b82a-d4cbdebf5de1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:44:12.181458 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86b68b-0111-4e91-b82a-d4cbdebf5de1 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:44:12.185279 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e7fda7-20a6-4979-9c35-465bb1066982 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd13a1638b2ca497fb02c71615559d1ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:44:12.185901 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7fda7-20a6-4979-9c35-465bb1066982 service neutron] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:44:12.186185 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7fda7-20a6-4979-9c35-465bb1066982 service neutron] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:44:12.186491 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7fda7-20a6-4979-9c35-465bb1066982 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:44:12.195691 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e7fda7-20a6-4979-9c35-465bb1066982 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:44:12.197184 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e7fda7-20a6-4979-9c35-465bb1066982 service neutron] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:44:12.207782 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1436/2894] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:44:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:44:12.797173 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e1a035-3bfb-40a8-b4f6-0625ca009220 None None] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:44:12.798959 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-d9e1a035-3bfb-40a8-b4f6-0625ca009220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:44:12.818275 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e1a035-3bfb-40a8-b4f6-0625ca009220 None None] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:44:12.826513 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18262c8b-3e1d-44a9-a538-066c52e94b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:44:12.827263 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18262c8b-3e1d-44a9-a538-066c52e94b28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:44:12.827623 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18262c8b-3e1d-44a9-a538-066c52e94b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:44:12.828135 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18262c8b-3e1d-44a9-a538-066c52e94b28 admin admin] PATH_INFO: `/v3/projects/6b22abde51b34b90a6601f1dbfda6f3e` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:44:12.829741 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18262c8b-3e1d-44a9-a538-066c52e94b28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b22abde51b34b90a6601f1dbfda6f3e)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:44:12.831925 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18262c8b-3e1d-44a9-a538-066c52e94b28 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:44:12.993358 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-18262c8b-3e1d-44a9-a538-066c52e94b28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b22abde51b34b90a6601f1dbfda6f3e', 'request_id': 'req-18262c8b-3e1d-44a9-a538-066c52e94b28'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:13.002800 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-18262c8b-3e1d-44a9-a538-066c52e94b28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b22abde51b34b90a6601f1dbfda6f3e', 'request_id': 'req-18262c8b-3e1d-44a9-a538-066c52e94b28'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:13.008637 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-18262c8b-3e1d-44a9-a538-066c52e94b28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b22abde51b34b90a6601f1dbfda6f3e', 'request_id': 'req-18262c8b-3e1d-44a9-a538-066c52e94b28'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:13.139213 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-18262c8b-3e1d-44a9-a538-066c52e94b28 admin admin] The token cache is being invalidate because project 6b22abde51b34b90a6601f1dbfda6f3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:44:13.139921 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-18262c8b-3e1d-44a9-a538-066c52e94b28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:13.140977 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-18262c8b-3e1d-44a9-a538-066c52e94b28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:13.142357 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1459/2895] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:44:12 2022] DELETE /identity/v3/projects/6b22abde51b34b90a6601f1dbfda6f3e => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:44:13.146403 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e7fda7-20a6-4979-9c35-465bb1066982 service neutron] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:44:13.153983 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-a1e7fda7-20a6-4979-9c35-465bb1066982 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:44:13.181364 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e7fda7-20a6-4979-9c35-465bb1066982 service neutron] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:44:13.247356 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619c954-2568-48a9-8b8e-226948ed83a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:44:13.248109 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619c954-2568-48a9-8b8e-226948ed83a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:44:13.248402 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619c954-2568-48a9-8b8e-226948ed83a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:44:13.248683 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619c954-2568-48a9-8b8e-226948ed83a1 admin admin] PATH_INFO: `/v3/users/f2a7a3b126294571a8586b6e7b43056c` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:44:13.250504 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e619c954-2568-48a9-8b8e-226948ed83a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2a7a3b126294571a8586b6e7b43056c)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:44:13.252540 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e619c954-2568-48a9-8b8e-226948ed83a1 admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:44:13.255476 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:44:13.446314 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-e619c954-2568-48a9-8b8e-226948ed83a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2a7a3b126294571a8586b6e7b43056c'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:13.453489 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-e619c954-2568-48a9-8b8e-226948ed83a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2a7a3b126294571a8586b6e7b43056c'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:13.454263 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-e619c954-2568-48a9-8b8e-226948ed83a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2a7a3b126294571a8586b6e7b43056c'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:13.466257 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-e619c954-2568-48a9-8b8e-226948ed83a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2a7a3b126294571a8586b6e7b43056c'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:13.466881 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-e619c954-2568-48a9-8b8e-226948ed83a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2a7a3b126294571a8586b6e7b43056c'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:13.523330 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1437/2896] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:44:13 2022] DELETE /identity/v3/users/f2a7a3b126294571a8586b6e7b43056c => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:44:13.782851 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18262c8b-3e1d-44a9-a538-066c52e94b28 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:44:13.784827 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-18262c8b-3e1d-44a9-a538-066c52e94b28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:44:13.800207 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18262c8b-3e1d-44a9-a538-066c52e94b28 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:44:13.856799 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f63984-b67e-4433-a249-368312f8d280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:44:13.857437 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f63984-b67e-4433-a249-368312f8d280 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:44:13.857722 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f63984-b67e-4433-a249-368312f8d280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:44:13.858002 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f63984-b67e-4433-a249-368312f8d280 admin admin] PATH_INFO: `/v3/projects/7b3f679565d546ff8e6418643699c002` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:44:13.859673 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f63984-b67e-4433-a249-368312f8d280 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b3f679565d546ff8e6418643699c002)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:44:13.861606 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f63984-b67e-4433-a249-368312f8d280 admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:44:13.878199 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:44:13.986892 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-30f63984-b67e-4433-a249-368312f8d280 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b3f679565d546ff8e6418643699c002', 'request_id': 'req-30f63984-b67e-4433-a249-368312f8d280'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:13.993197 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-30f63984-b67e-4433-a249-368312f8d280 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b3f679565d546ff8e6418643699c002', 'request_id': 'req-30f63984-b67e-4433-a249-368312f8d280'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:13.998908 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-30f63984-b67e-4433-a249-368312f8d280 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b3f679565d546ff8e6418643699c002', 'request_id': 'req-30f63984-b67e-4433-a249-368312f8d280'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:14.053783 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-30f63984-b67e-4433-a249-368312f8d280 admin admin] The token cache is being invalidate because project 7b3f679565d546ff8e6418643699c002 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:44:14.054079 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-30f63984-b67e-4433-a249-368312f8d280 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:14.054709 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-30f63984-b67e-4433-a249-368312f8d280 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:14.056260 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1460/2897] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:44:13 2022] DELETE /identity/v3/projects/7b3f679565d546ff8e6418643699c002 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:44:14.060367 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619c954-2568-48a9-8b8e-226948ed83a1 admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:44:14.061776 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-e619c954-2568-48a9-8b8e-226948ed83a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:44:14.074895 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619c954-2568-48a9-8b8e-226948ed83a1 admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:44:14.131339 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7dbbb2-1836-4749-95af-5857232268ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:44:14.131991 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7dbbb2-1836-4749-95af-5857232268ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:44:14.132309 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7dbbb2-1836-4749-95af-5857232268ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:44:14.132591 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7dbbb2-1836-4749-95af-5857232268ba admin admin] PATH_INFO: `/v3/users/d14cd2362c3b4ccebf8403728b87d020` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:44:14.133780 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7dbbb2-1836-4749-95af-5857232268ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=d14cd2362c3b4ccebf8403728b87d020)` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:44:14.135689 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7dbbb2-1836-4749-95af-5857232268ba admin admin] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:44:14.143648 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f63984-b67e-4433-a249-368312f8d280 admin admin] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:44:14.145076 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-30f63984-b67e-4433-a249-368312f8d280 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:44:14.215823 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f63984-b67e-4433-a249-368312f8d280 admin admin] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:44:14.221488 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b57fd7-ddd1-4001-b2a5-c0e8edac80c6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '107a7c18a2c6440b86e1b7c3e39cc724', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:44:14.222098 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b57fd7-ddd1-4001-b2a5-c0e8edac80c6 service placement] REQUEST_METHOD: `GET` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:44:14.222379 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b57fd7-ddd1-4001-b2a5-c0e8edac80c6 service placement] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:44:14.222653 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b57fd7-ddd1-4001-b2a5-c0e8edac80c6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:44:14.224429 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-06b57fd7-ddd1-4001-b2a5-c0e8edac80c6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:44:14.236438 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b57fd7-ddd1-4001-b2a5-c0e8edac80c6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:44:14.238110 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b57fd7-ddd1-4001-b2a5-c0e8edac80c6 service placement] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:44:14.248159 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1461/2898] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:44:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:44:14.329740 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-df7dbbb2-1836-4749-95af-5857232268ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd14cd2362c3b4ccebf8403728b87d020'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:14.336585 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-df7dbbb2-1836-4749-95af-5857232268ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd14cd2362c3b4ccebf8403728b87d020'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:14.337288 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-df7dbbb2-1836-4749-95af-5857232268ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd14cd2362c3b4ccebf8403728b87d020'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:14.349635 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-df7dbbb2-1836-4749-95af-5857232268ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd14cd2362c3b4ccebf8403728b87d020'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:14.354104 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.notifications [None req-df7dbbb2-1836-4749-95af-5857232268ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd14cd2362c3b4ccebf8403728b87d020'} {{(pid=80541) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:14.451764 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:44:14.459713 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1438/2899] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 5 15:44:14 2022] DELETE /identity/v3/users/d14cd2362c3b4ccebf8403728b87d020 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:44:14.689151 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b57fd7-ddd1-4001-b2a5-c0e8edac80c6 service placement] Authenticating user token {{(pid=80542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:44:14.690960 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.fernet_utils [None req-06b57fd7-ddd1-4001-b2a5-c0e8edac80c6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:44:14.710425 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b57fd7-ddd1-4001-b2a5-c0e8edac80c6 service placement] Validating token access rules against request {{(pid=80542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:44:14.768277 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be62dc-9079-4f9f-887e-fb24642e419c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '338271833df8486b9ef50964b5ec7797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0744d3ce240a4f198b94f2d1e6d62c8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:44:14.768924 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be62dc-9079-4f9f-887e-fb24642e419c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:44:14.769212 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be62dc-9079-4f9f-887e-fb24642e419c admin admin] SCRIPT_NAME: `/identity` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:44:14.769493 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be62dc-9079-4f9f-887e-fb24642e419c admin admin] PATH_INFO: `/v3/projects/5bf684650c2145cbb6d131625235ce7e` {{(pid=80542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:44:14.770660 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be62dc-9079-4f9f-887e-fb24642e419c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bf684650c2145cbb6d131625235ce7e)` {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:44:14.772606 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be62dc-9079-4f9f-887e-fb24642e419c admin admin] RBAC: Authorization granted {{(pid=80542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:44:14.875715 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:44:14.884284 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-38be62dc-9079-4f9f-887e-fb24642e419c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bf684650c2145cbb6d131625235ce7e', 'request_id': 'req-38be62dc-9079-4f9f-887e-fb24642e419c'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:14.890957 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-38be62dc-9079-4f9f-887e-fb24642e419c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bf684650c2145cbb6d131625235ce7e', 'request_id': 'req-38be62dc-9079-4f9f-887e-fb24642e419c'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:14.896472 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-38be62dc-9079-4f9f-887e-fb24642e419c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bf684650c2145cbb6d131625235ce7e', 'request_id': 'req-38be62dc-9079-4f9f-887e-fb24642e419c'} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:14.968914 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-38be62dc-9079-4f9f-887e-fb24642e419c admin admin] The token cache is being invalidate because project 5bf684650c2145cbb6d131625235ce7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 15:44:14.969208 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-38be62dc-9079-4f9f-887e-fb24642e419c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:14.969843 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG keystone.notifications [None req-38be62dc-9079-4f9f-887e-fb24642e419c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 15:44:14.971361 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: [pid: 80542|app: 0|req: 1462/2900] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 5 15:44:14 2022] DELETE /identity/v3/projects/5bf684650c2145cbb6d131625235ce7e => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 15:44:25.021319 dv-cinder-849890-4-iscsi devstack@keystone.service[80542]: DEBUG dbcounter [-] [80542] Writing DB stats keystone:SELECT=11,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80542) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 15:49:17.171871 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7dbbb2-1836-4749-95af-5857232268ba admin admin] Authenticating user token {{(pid=80541) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 15:49:17.173446 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-df7dbbb2-1836-4749-95af-5857232268ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:49:17.311729 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7dbbb2-1836-4749-95af-5857232268ba admin admin] Validating token access rules against request {{(pid=80541) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 15:49:17.371239 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e94fb-b7ac-4bd4-8145-cbf76e697d80 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '107a7c18a2c6440b86e1b7c3e39cc724', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06f27ab8886a4c0d81c4d4e3fa100a83', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80541) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 15:49:17.375575 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e94fb-b7ac-4bd4-8145-cbf76e697d80 service placement] REQUEST_METHOD: `GET` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 15:49:17.375575 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e94fb-b7ac-4bd4-8145-cbf76e697d80 service placement] SCRIPT_NAME: `/identity` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 15:49:17.375575 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e94fb-b7ac-4bd4-8145-cbf76e697d80 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80541) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 15:49:17.375575 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.fernet_utils [None req-8a4e94fb-b7ac-4bd4-8145-cbf76e697d80 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80541) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 15:49:17.386963 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4e94fb-b7ac-4bd4-8145-cbf76e697d80 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 15:49:17.388875 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4e94fb-b7ac-4bd4-8145-cbf76e697d80 service placement] RBAC: Authorization granted {{(pid=80541) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 15:49:17.399673 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: [pid: 80541|app: 0|req: 1439/2901] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 5 15:49:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 15:49:27.396006 dv-cinder-849890-4-iscsi devstack@keystone.service[80541]: DEBUG dbcounter [-] [80541] Writing DB stats keystone:SELECT=89 {{(pid=80541) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}